Tracking Progress: Setting, Collecting, and Reflecting on Implementation Markers

FavoriteTracking progress is an important component of the Hewlett Foundation’s practice of outcome-focused philanthropy (OFP), which emphasizes being rigorous, flexible, and adaptive while staying focused on results and actively learning throughout the strategy lifecycle. The outcomes we hope to achieve through our strategies take years to accomplish. Implementation markers represent interim steps toward these outcomes. “Implementation marker” is a catchall term referring to particular activities, developments or events (internal or external) that help staff understand whether they are making progress in the near term. This guide is a deep dive into implementation markers—what they are and how to set, collect, Read more

PCI: A Reflective Evaluation Framework for Systems Change

FavoriteThis article explores the interconnected dimensions of the PCI Reflective Evaluation Framework, an approach now in prototype form which is grounded in practical thinking about working within complex social systems. This article focuses on its use in advancing racial equity, describing possible applications to integrate a racial equity lens in unpacking and addressing the complexity of systemic change. The framework is intended to help communities use evaluation to sustain their efforts to achieve racial equity and other systemic change goals that involve fundamental shifts in the underlying assumptions and values on which a social system is built.

Marguerite Casey Foundation: Reflecting on 15 Years of Philanthropic Leadership Through a Summative Evaluation

FavoriteThis article presents the findings of a summative evaluation of the Marguerite Casey Foundation that was conducted on the occasion of its 15th anniversary. The evaluation was designed to gauge stakeholders’ perceptions of the foundation’s operations to facilitate organizational learning. In sharing these results, the authors seek to elucidate the role of evaluation as a learning practice within the field of philanthropy.

Foundations as Network Strategists, Weavers, and Managers: Learning From One Foundation’s Journey and Results

FavoriteThis article shares insights from a five-year evaluation of the Oral Health 2020 network, an effort by the DentaQuest Foundation to align and strengthen efforts in service of a national movement to improve oral health. The evaluation helped to place the foundation’s journey in the context of a broader field seeking new approaches to achieve deep and sustainable social change.


6 Steps to Evaluating your Program

Favorite One of the most effective things you can do to strengthen your dialogue-to-change program is to create an accurate process for documenting and evaluating the entire effort. Many people who take part in your program will want to know who participated, how effective the organizing strategies were, and what the outcomes were. Moreover, grant-making foundations, public officials, news media, and other people who can help you expand, strengthen, and institutionalize the dialogue-to-change program in your community will all want to know about your efforts and their impact. 1. Revisit your goals 2. Set benchmarks 3. Document the basics 4. Track Read more

End-Game Evaluation: Building a Legacy of Learning In a Limited-Life Foundation

Favorite This article shares the emerging hypotheses of two foundations, The Atlantic Philanthropies and the S. D. Bechtel, Jr. Foundation — each four years from sunset — about the opportunities and challenges for evaluation in the limited-life context. The article argues that systematically capturing and sharing knowledge — about programs, as well as social-change methods and grantmaking practices — can increase a foundation’s influence and impact during its final years and beyond.

Exiting From Large-Scale Initiatives: Lessons and Insights From a National Scan of Philanthropy

Favorite This article shares insights and lessons from a research project commissioned by The California Endowment in early 2016 to inform the planning for its transition out of Building Healthy Communities. A guiding framework for exit and sustainability planning is presented as a set of recommendations that relate to issues such as managing relationships between funder and grantee partners during the exit, using the initiative’s theory of change as a tool for decision-making, finding a balance between demonstrable success and equity, and managing the internal processes of the funding organization.

Breaking Up Is Hard to Do

FavoriteThis article discusses four areas where foundation exits present particular challenges and where there are significant opportunities to improve practice — deciding on and planning to exit, funder leadership, clear communication, and final grants — and includes summaries of advice from funder and grantee perspectives.

35 Keys to Effective Evaluation

FavoriteAs the need for scarce grant dollars grows more intense, so does the need to make certain those dollars are spent as effectively as possible. Hence the question of how to evaluate the consequences of grant supported activities has risen to the forefront. This tool will help you make sure grant money is spent as effectively as possible and how to evaluate the consequences of grant supported activities  

The Legacy of a Philanthropic Exit: Lessons From the Evaluation of the Hewlett Foundation’s Nuclear Security Initiative

Favorite This article describes how the findings of the NSI evaluation informed Hewlett’s philanthropic approach, and provides a case example of a philanthropic-initiative exit. Key considerations for monitoring and evaluation practices particular to the context of a planned exit are discussed.

Benchmarking Foundation Evaluation Practices

FavoriteDeveloped in partnership with the Center for Evaluation Innovation, Benchmarking Foundation Evaluation Practices is the most comprehensive data collection effort to date on evaluation practices at foundations. The report shares data points and infographics on crucial topics related to evaluation at foundations, such as evaluation staffing and structures, investment in evaluation work, and the usefulness of evaluation information.

Evaluating Complexity

FavoriteWhile evaluation has traditionally focused on assessing programmatic impact according to pre-determined indicators, a new approach is needed for evaluating complex initiatives, as well as initiatives operating in complex environments where progress is not linear, predictable, or controllable. 9 propositions can help evaluators navigate the unique characteristics of complex systems, improve their evaluation practice, and better serve the needs of the social sector.

Benchmarking Evaluation in Foundations: Do We Know What We Are Doing?

FavoriteThis article presents new findings about what foundations are doing on evaluation and discusses their implications. It is based on 2012 research that benchmarks the positioning, resourcing, and function of evaluation in foundations, and follows up on a 2009 study that used a similar design.  

Evaluating the Kaiser Permanente Community Fund’s Social Determinants of Health Portfolio

FavoriteThe authors present an overview of the Kaiser Permanente Community Fund’s social determinants of health initiative and its theory of change. The fund is based at the Northwest Health Foundation. The authors introduce frameworks and methods used to conduct their evaluation. The fund reached multiple sectors and established new partners and relationships, but the lack of depth may limit opportunities to make a profound and measurable difference within any specic domain.

Defining, Building, and Measuring Capacity: Findings From an Advocacy Evaluation

FavoriteThis article reports the results from Mathematica Policy Research’s evaluation of Consumer Voices for Coverage, a program funded by the Robert Wood Johnson Foundation to support the role of consumer health advocacy coalitions in 12 states. The authors propose that funders address three elements of coalition capacity: knowledge, infrastructure, and resources. Each requires different types of interventions.

Evaluation for Models and Adaptive Initiatives

FavoriteThe authors concentrate on how the nature of the intervention affects evaluation design. They outline a framework for selecting evaluation approaches for two types of grantmaking programs used to achieve far-reaching impact: models and adaptive initatives. Evaluation that is attuned to the transformations in models and adaptive initiatives will continue to help fuel these two powerful engines of social change.

Can Feedback Fuel Change at Foundations?: An Analysis of the Grantee Perception Report

FavoriteThis brief report presents key findings about Robert Wood Johnson Foundation’s Grantee Perception Reports over the years, and includes candid comments by RWJF’s leadership — CEO Risa Lavizzo-Mourey and Vice President of Research and Evaluation, David Colby — as to how they’ve responded to some of those findings.

Evaluative Tools for Articulating and Monitoring Foundation Strategy

FavoriteWhile foundation leadership and staff value strategy and foundations largely perceive themselves as strategic, they often struggle to articulate, implement, and track strategy. The William Penn Foundation has developed a collection of tools to articulate and assess its progress toward strategic goals.

Foundation Evaluation Startup: A Pause for Reflection

FavoriteThis article reports on the accomplishments, challenges, and lessons learned in creating a new Department of Research and Evaluation at the California HealthCare Foundation. Different tools were developed to address each of three key areas: performance assessment, organizational learning, and program evaluation.      

Learning-Circle Partnerships and the Evaluation of a Boundary-Crossing Leadership Initiative in Health

FavoriteThis article discusses learing-circles, a multicultural approach to evaluation that honors different ways of knowing, recognizes that groups have different learning questions, acknowledges and addresses power dynamics that exist between funders and grantees, and ensures that evaluation is culturally relevant and constructive for communities.  

The Real-Time Evaluation Memo: A Tool for Enabling Evaluative Thinking and Learning in Foundations and Nonprofits

FavoriteThis article tells how real-time evaluation memos provide data-based feedback in a timely manner to inform decision making. Memos must be concise and include both data and expert synthesis and interpretation. The foundation must have a learning culture if the memos are to most useful; there must be time to reflect on the content and implications.  

Your Toolkit for Successful Evaluations

Favorite This resource has been designed and assembled by the Macro International evaluation team with the support of the William and Flora Hewlett Foundation as a starter kit to promote the integration of effective evaluation into grantmaker education programs. This core set of tools includes general tips for conducting successful evaluations, guidance and examples for defining and understanding a program’s logic, how-to instructions for designing participant questionnaires, and an annotated guide to existing evaluation resources to support good practice.

Pathfinder Funder Edition: A Practical Guide to Advocacy Evaluation

FavoritePathfinder is a practical guide to the advocacy evaluation process. This edition guides funders through the advocacy evaluation process from start to finish. Editions for advocates and evaluators are also available. Drawn from Innovation Network’s research and consulting experience, Pathfinder encourages the adoption of a “learning-focused evaluation” approach, which prioritizes using knowledge for improvement.

Are We There Yet? How to Know Whether Your Communications Are Effective

FavoriteThis article describes the results of a study on current knowledge and practices in evaluating foundation communications. The study consisted of three parts: an online survey of practitioners, a series of in-depth key informant interviews, and an extensive literature review.

Going Deeper: Can Investigative Reporters Add Value to Assessment and Evaluation?

FavoriteThe John S. and James L. Knight Foundation supplemented its standard evaluation approach by engaging professional journalists to elaborate on evaluation findings. The resulting reports are more direct, even critical, than any prior Knight Foundation attempt to evaluate and assess. It produced deeper looks into the intent and outcome of major initiatives, analyzing and addressing flaws in the theories of change underlying initiatives.  

Using Community-Based Participatory Evaluation (CBPE) Methods as a Tool to Sustain a Community Health Coalition

FavoriteParticipatory evaluation has set the standard for cooperation between program evaluators and stakeholders. Coalition evaluation, however, calls for more extensive collaboration with the community at large. There are many different labels for participatory evaluation, however not one specific to coalition work. This article describes Community-Based Participatory Evaluation CBPE, takes time, money and skilled personnel but can lead to more accurate results and coalition sustainability. Recommendations on how foundations and grantee organizations interested in coalition work should fund CBPE are discussed.

Philanthropy, Evaluation, Accountability, and Social Change

FavoriteThe author argues that many foundations have substituted process accountability for accountability for contributing to social change. Accountability in terms of required reporting is important, but it sets a floor, not an aspirational ceiling. There are tools — such as risk analysis, systems approaches, and game theory — that can help philanthropy engage in work on complex social problems that cannot be deconstructed into a series of small, linear projects.

Youth Civic Engagement for Dialogue and Diversity at the Metropolitan Level

FavoriteThis article analyzes Youth Dialogues on Race and Ethnicity, a foundation-funded program designed to increase dialogue, challenge segregation, and create change in metropolitan Detroit. It draws on multilevel evaluation of the program and analyzes some of the lessons learned.

A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions

FavoriteRWJF commissioned FSG Social Impact Advisors to develop a guide for program officers, grant recipients, evaluators, researchers, and others interested in evaluation on how to engage stakeholders in developing evaluation questions. Since stakeholders are potential users of evaluation findings, their input into the scope of the evaluation is critical to ensuring the integrity and value of evaluation results. This guide provides the reader with a five-step process for involving stakeholders in developing evaluation questions, and includes a set of four worksheets to facilitate this process.

The Robert Wood Johnson Foundation’s Approach to Evaluation

FavoriteThe Robert Wood Johnson Foundation has placed a high priority on program evaluation since its inception as a national philanthropy in 1972. It has developed a four-tiered system of evaluation that ranges from the evaluation of individual grants and clusters of grants to the qualitative assessments found in The Robert Wood Johnson Foundation Anthology series.