0

The Philanthropy As One Big Impact Investment: A Framework For Evaluating A Foundation’s Blended Performance

This article proposes a framework for evaluating a foundation’s blended performance that enables both grantmaking and endowment investing to be evaluated jointly, and thus also allows a complete evaluation of how impact investments could improve — or fail to improve — overall performance.





0

Marguerite Casey Foundation: Reflecting on 15 Years of Philanthropic Leadership Through a Summative Evaluation

This article presents the findings of a summative evaluation of the Marguerite Casey Foundation that was conducted on the occasion of its 15th anniversary. The evaluation was designed to gauge stakeholders’ perceptions of the foundation’s operations to facilitate organizational learning. In sharing these results, the authors seek to elucidate the role of evaluation as a learning practice within the field of philanthropy.





0

Foundations as Network Strategists, Weavers, and Managers: Learning From One Foundation’s Journey and Results

This article shares insights from a five-year evaluation of the Oral Health 2020 network, an effort by the DentaQuest Foundation to align and strengthen efforts in service of a national movement to improve oral health. The evaluation helped to place the foundation’s journey in the context of a broader field seeking new approaches to achieve deep and sustainable social change.









0

Changing in Place: The Skillman Foundation, Detroit, and the Good Neighborhoods Initiative — How did a hometown grantmaker conduct and conclude its largest-ever initiative?

The initiative concluded in 2016, ultimately spanning 11 years and involving $122 million in grants, which represented 67 percent of the Foundation’s total grant spending in this time frame. Along the way, the foundation reset its strategy and sharpened its goal — in response to seismic shifts in the local context and informed by indicators of progress. To capture information on the unique challenges facing an embedded funder as it changes program direction, Bob Tobin, senior consultant at Williams Group, interviewed Marie Colombo, Skillman Foundation director for strategic evaluation and learning.





0

End-Game Evaluation: Building a Legacy of Learning In a Limited-Life Foundation

Few, if any, of the problems philanthropy seeks to address can be solved within a brief, defined time frame. Limited-life foundations can only strive to move the ball down the field before they sunset, and then enlist others to carry the work forward. This article shares the emerging hypotheses of two foundations, The Atlantic Philanthropies and the S. D. Bechtel, Jr. Foundation — each four years from sunset — about the opportunities and challenges for evaluation in the limited-life context. The article argues that systematically capturing and sharing knowledge — about programs, as well as social-change methods and grantmaking practices Read more





0

Exiting From Large-Scale Initiatives: Lessons and Insights From a National Scan of Philanthropy

This article shares insights and lessons from a research project commissioned by The California Endowment in early 2016 to inform the planning for its transition out of Building Healthy Communities. A guiding framework for exit and sustainability planning is presented as a set of recommendations that relate to issues such as managing relationships between funder and grantee partners during the exit, using the initiative’s theory of change as a tool for decision-making, finding a balance between demonstrable success and equity, and managing the internal processes of the funding organization.





0

Breaking Up Is Hard to Do

Funding relationships begin, and they end. Yet little is known about the effects of foundation exits on the work, the grantees, and the related fields. This article draws on interviews with funders and grantees involved in more than a dozen exits to fill the gaps in what is known about how to exit well. The article discusses four areas where foundation exits present particular challenges and where there are significant opportunities to improve practice — deciding on and planning to exit, funder leadership, clear communication, and final grants — and includes summaries of advice from funder and grantee perspectives.







0

The Legacy of a Philanthropic Exit: Lessons From the Evaluation of the Hewlett Foundation’s Nuclear Security Initiative

Although time-bound philanthropic initiatives are a well-established practice, there is still much to learn about effective ways to implement, evaluate, and wind down these types of investments. This open-access article describes the NSI evaluation, how the findings informed Hewlett’s philanthropic approach, and provides a case example of a philanthropic-initiative exit. Key considerations for monitoring and evaluation practices particular to the context of a planned exit are discussed.







0

Mapping the Journey to Impact Investing

This report charts the journey from the time the Surdna Foundation board of directors and staff began exploring impact investing in 2014 to the decision-making process and experience of implementing impact investing policies. “The Surdna Foundation’s founder, John E. Andrus was committed to inclusion, social justice and sustainability,” said Peter Benedict II, Surdna Foundation’s board chair. “By sharing our experience and some of the lessons we learned in this report this centennial year, we will contribute to collective learning in the fields of mission-related investing and family philanthropy and celebrate these core values.”









0

Benchmarking Foundation Evaluation Practices

Developed in partnership with the Center for Evaluation Innovation (CEI), Benchmarking Foundation Evaluation Practicesis the most comprehensive data collection effort to date on evaluation practices at foundations. The report shares data points and infographics on crucial topics related to evaluation at foundations, such as evaluation staffing and structures, investment in evaluation work, and the usefulness of evaluation information.





0

Evaluating Complexity

While evaluation has traditionally focused on assessing programmatic impact according to pre-determined indicators, a new approach is needed for evaluating complex initiatives, as well as initiatives operating in complex environments where progress is not linear, predictable, or controllable. 9 propositions can help evaluators navigate the unique characteristics of complex systems, improve their evaluation practice, and better serve the needs of the social sector.





0

Strengthening Nonprofit Capacity

Capacity building enables nonprofit leaders and organizations to develop the skills and resources they need to improve their work. Since each situation is unique and circumstances are always changing, effective capacity-building support is tailored to best suit the needs of grantees. This publication offers practical guidance and considerations to help grantmakers design an impactful approach.





0

Supporting Social Entrepreneurship

For philanthropists accustomed to more traditional grantees, yet interested in backing these groundbreaking leaders, funding social entrepreneurs can feel like entering uncharted territory. Donors might find themselves intrigued by the potential for change, and yet, at the same time, unsure of what to expect in a field where the unexpected is the norm. That’s why we wrote this brief guide. Think of it as an introduction to social entrepreneurship. Part of our Philanthropy Roadmap series, the guide is designed to help philanthropists evaluate whether they want to include support for social entrepreneurs in their giving or investment programs, and how Read more







0

Evaluating the Kaiser Permanente Community Fund’s Social Determinants of Health Portfolio

Research over the past two decades repeatedly demonstrates the relationship between poor health outcomes and socioeconomic factors such as poor housing, poverty, racism, and structural inequity. In 2005, the Northwest Health Foundation, supported by the Kaiser Permanente Community Fund, began an initiative to address these social determinants of health (SDOH). A variety of projects – short- and long-term, large and small – were supported over the five-year period for a total of $12.4 million. The mean project-implementation grant was $175,350 and 2½ years in length; capacity-building grants averaged $50,000 for 1½ years. In all, 323 social-determinant accomplishments were identified. The Read more





0

Defining, Building, and Measuring Capacity: Findings From an Advocacy Evaluation

This article reports on results from Mathematica Policy Research’s evaluation of Consumer Voices for Coverage, a program funded by the Robert Wood Johnson Foundation to support the role of consumer health advocacy coalitions in 12 states. The foundation based the program on a study that identified six core advocacy capacities, and designed it to strengthen these capacities. The evaluation found that the level of funding, substantial and targeted technical assistance, and the three-year time frame of the program contributed to the observed increases in five capacities. Fundraising remained the lowest-rated capacity for most of the coalitions and may require different Read more





0

Evaluation for Models and Adaptive Initiatives

Although there has been a growing emphasis on use of experimental designs in evaluation, there is also increasing agreement that evaluation designs should be situation specific. The nature of the program is one of the key factors to consider in evaluation design. Two types of programs – models, which provide replicable or semi-standardized solutions, adaptive initiatives, which are flexible programming strategies used to address problems that require unique, context-based solutions – require different evaluation designs. Evaluation of models requires understanding the stage of development of the model program, with summative evaluation done only when the model is fully developed. Adaptive Read more





0

Sustainability Is Made, Not Born: Enhancing Program Sustainability Through Reflective Grantmaking

This article explores how reflective grantmaking can lead to enduring changes in the communities that foundations serve. The Health Foundation of Greater Cincinnati’s approach to evaluating and improving the sustainability of grant-funded projects is reviewed as an example. Their grantmaking framework includes policy and advocacy work, evaluation support, communications support, and technical assistance in addition to traditional funding of projects. This framework promotes sustainability of the funded work.





0

Designing an Evaluation of a New Initiative: A Practical Approach to Ensure Evaluation Use

This article describes the process of planning an evaluation of the Tribal Tobacco Education and Policy initiative. The initiative was launched in 2007 to reduce tobacco use among American Indians, who disproportionately suffer the negative health effects of tobacco use. The work of the initiative and the evaluation had to incorporate an understanding of tribal structure as well as of the traditional use of tobacco in American Indian sacred ceremonies. The theory of change was conceptualized as circular, rather than linear, in keeping with American Indian philosophical traditions. The planning process, utilizing evaluators familiar with community mobilization and policy evaluation Read more





0

From Insight to Action: New Directions in Foundation Evaluation

After a year of research and nearly 100 interviews with foundation leaders and evaluation experts, FSG has released a report that identifies a fundamental transition in the way foundations use evaluation. The study was funded by the William & Flora Hewlett Foundation, and conducted in collaboration with the Forum of Regional Associations of Grantmakers.









0

Getting to Impact Through Evaluation

This primer introduces you to simple, straightforward evaluation from start to finish so you can: Set the stage for evaluation at your foundation. Create a road map for your giving that will help you target an evaluation. Understand five key steps of evaluation: targeting, asking, tracking, learning, and using. Know where to turn for support when evaluating your work. Get started on the road toward even greater impact.





0

The Grantmaking School: Evaluation for Grantmakers

Effective evaluation at your foundation can help you overcome those challenges with new solutions for your work and that of your grantees. The Grantmaking School’s Evaluation for Grantmakers gives you the opportunity to examine the history, strategy, design, and theory behind the process of evaluation and get equipped with the tools you need to respond to the diverse nature of evaluation in foundations.





0

The Robert Wood Johnson Foundation’s Approach to Evaluation

The Robert Wood Johnson Foundation has placed a high priority on program evaluation since its inception as a national philanthropy in 1972. It has developed a four-tiered system of evaluation that ranges from the evaluation of individual grants and clusters of grants to the qualitative assessments found in The Robert Wood Johnson Foundation Anthology series.





0

Guidance on Evaluation Reports to the Robert Wood Johnson Foundation: A Checklist for Evaluators

The Robert Wood Johnson Foundation (RWJF) is a long-time proponent of evaluation as a means to inquire systematically into the effects and impacts of its grantmaking programs. Evaluation reports to RWJF should generally include the elements listed in this checklist to be considered satisfactory. The checklist was adapted from several references listed at the bottom of the document.





0

Consumer Voices for Coverage: Advocacy Evaluation Toolkit

The Advocacy Evaluation Toolkit contains the instruments Mathematica used to collect data for evaluating the Consumer Voices for Coverage program. It explains how the instruments were developed, what each was designed to measure, and how Mathematica used them for the evaluation. Although the instruments in the toolkit were designed to collect data for the grant program and reflect its structure and goals, they can be adapted for other situations and uses, ranging from an organization’s informal self-assessment to shape its activities to a comprehensive evaluation. The toolkit suggests some of these adaptations. For people who might not be familiar with evaluation methods, Read more





0

Can Feedback Fuel Change at Foundations?: An Analysis of the Grantee Perception Report

This brief report presents key findings about Robert Wood Johnson Foundation’s  Grantee Perception Reports over the years, and includes candid comments by RWJF’s leadership—CEO Risa Lavizzo-Mourey and Vice President of Research and Evaluation, David Colby—as to how they’ve responded to some of those findings.





0

An Ecological Understanding of Evaluation Use

This report seeks to be helpful not only to the Robert Wood Johnson Foundation and other foundations, but more broadly to the fields of evaluation, aging, physical activity and public health in understanding what evaluation use looks like in context and how it can be facilitated. This case study illustrates the parameters of use (e.g., for whom? when? how?), as well as what constituted the use of the evaluation of the Robert Wood Johnson Foundation (RWJF) program Active for Life®: Increasing Physical Activity Levels in Adults Age 50 and Older.





0

A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions

RWJF commissioned FSG Social Impact Advisors to develop a guide for program officers, grant recipients, evaluators, researchers, and others interested in evaluation on how to engage stakeholders in developing evaluation questions. Since stakeholders are potential users of evaluation findings, their input into the scope of the evaluation is critical to ensuring the integrity and value of evaluation results. This guide provides the reader with a five-step process for involving stakeholders in developing evaluation questions, and includes a set of four worksheets to facilitate this process.









0

Pathfinder Funder Edition: A Practical Guide to Advocacy Evaluation

Pathfinder is a practical guide to the advocacy evaluation process. This edition guides funders through the advocacy evaluation process from start to finish. Editions for advocates and evaluators are also available. Drawn from Innovation Network’s research and consulting experience, Pathfinder encourages the adoption of a “learning-focused evaluation” approach, which prioritizes using knowledge for improvement.





0

Evaluation Capacity Building: Funder Initiatives to Strengthen Grantee Evaluation Capacity and Practice

The three case studies presented in this paper share Innovation Network’s experiences with funder initiatives to strengthen grantee evaluation capacity and practice. Each case study includes a description of the overall grantmaking initiative, followed by a discussion of the Evaluation Capacity Building (ECB) services requested by grantees and/or provided by Innovation Network. The paper concludes with a reflection on lessons learned and recommendations for funders considering ECB for their grantees.  











0

A Model for Multilevel Advocacy Evaluation

The Colorado Trust provided three years of general operating support to nine advocacy organizations working to increase access to health through policy change work. The nine grantees had a variety of goals and strategies and had different levels of organizational capacity, but were evaluated using a uniform evaluation approach. The evaluation was designed to build grantees’ own evaluation capacity to incorporate real-time feedback, monitor progress toward goals, and to assess growth in the overall health advocacy community in Colorado. Individual grantees identified short- and intermediate- term outcomes related to The Trust’s intermediate outcomes, which were in turn related to the Read more





0

A Road Made by Walking: Participatory Evaluation and Social Change

This article describes how participatory evaluation was used in a Ford Foundation–funded project to promote mixed-income housing in Atlanta. The project resulted in an increase in mixed income housing, but also in social outcomes such as increased knowledge about housing issues. Validity and reliability of the findings are demonstrated through feedback from the community members, rather than through statistical methods.





0

Are We There Yet? How to Know Whether Your Communications Are Effective

This article describes the results of a study on current knowledge and practices in evaluating foundation communications. The study consisted of three parts: an online survey of practitioners, a series of in-depth key informant interviews, and an extensive literature review. The study found that while most practitioners agree that evaluating communications is necessary to make decisions about their communication strategy, more than half did not regularly do so. Lack of experience or skills was the second top barrier cited, after lack of human/financial resources. Those who have more experience with evaluation were more likely to feel that it was not Read more





0

Evaluative Tools for Articulating and Monitoring Foundation Strategy

Research shows that while foundation leadership and staff value strategy and foundations largely perceive themselves as strategic, they often struggle to articulate, implement, and track strategy. The William Penn Foundation has developed a collection of tools to articulate and assess its progress toward strategic goals. Each tool employs a structured format to promote standardization; flexibility, though, is encouraged in the application of each tool to ensure that form does not dictate function. Each tool provides a template for organizing information that should be tweaked as needed. The speed and breadth of adoption of each tool varies and is often related Read more





0

Foundation Evaluation Startup: A Pause for Reflection

This article reports on the accomplishments, challenges, and lessons learned in creating a new Department of Research and Evaluation at the California HealthCare Foundation. Different tools were developed to address each of three key areas: performance assessment, organizational learning, and program evaluation. These new processes and tools have been well received by both staff and the board, and have become increasingly important as resources become more scarce, making understanding and maximizing the impact of investments even more critical. Fostering a culture of evaluative inquiry in a fast-paced, payout-oriented environment is a significant challenge – program staff often feels pressured to Read more





0

Going Deeper: Can Investigative Reporters Add Value to Assessment and Evaluation?

The John S. and James L. Knight Foundation supplemented its standard evaluation approach by engaging professional journalists to elaborate on evaluation findings. The resulting reports are more direct, even critical, than any prior Knight Foundation attempt to evaluate and assess. It produced deeper looks into the intent and outcome of major initiatives, analyzing and addressing flaws in the theories of change underlying initiatives. The goal of reaching external audiences was not achieved.





0

Leveraging Grant-Making—Part 2: Aligning Programmatic Approaches With Complex System Dynamics

The purpose of this two-part article is to enable foundations to increase the leverage of their grantmaking resources by working effectively with the dynamics of complex social systems. This article examines how foundations can align planning, implementation, and evaluation efforts with the behavior of the social systems they seek to improve. Asking powerful questions of staff, board, grantees, and other stakeholders helps to transform how they think about their goals and strategies. In addition to using the power of questioning, foundations function more systemically by suspending their assumptions about their effectiveness and what is possible, creating the cultural shifts needed, Read more







0

Using Community-Based Participatory Evaluation (CBPE) Methods as a Tool to Sustain a Community Health Coalition

Participatory evaluation has set the standard for cooperation between program evaluators and stakeholders. Coalition evaluation, however, calls for more extensive collaboration with the community at large. Integrating principles of community based participatory research and the Substance Abuse and Mental Health Services Administration’s Strategic Prevention Framework, which guides much coalition work, into coalition evaluation has proved useful to foster community affiliations and support reciprocal relationship building. The resulting evaluation method, named community based participatory evaluation (CBPE), takes time, money, and skilled personnel but can lead to more accurate results and coalition sustainability. The CBPE method has proved essential in sustaining two Read more





0

The Real-Time Evaluation Memo: A Tool for Enabling Evaluative Thinking and Learning in Foundations and Nonprofits

Real-time evaluation memos provide data-based feedback in a timely manner to inform decision making. Memos must be concise and include both data and expert synthesis and interpretation. The foundation must have a learning culture if the memos are to most useful; there must be time to reflect on the content and implications. The balance between data quality and timeliness must be managed and will be dependent on the topic. While useful for program management, these memos do not provide the kind of summative information that board members and other stakeholders may require.





0

The Quest for Quality: Lessons From a Certification Pilot Project for College Access Providers

Certification programs can provide a way for nonprofits to ensure adherence to generally accepted standards. The KH2GO Certification Pilot Project, supported by the Lumina Foundations, developed a set of standards for high-quality college access services, including standards for programming, operations, and organizational effectiveness. The project was implemented in two states with an evaluation designed to assess the quality of the assessment tools and the ease and rigor of implementation. The more clarity that applicants had about the goals of the process, potential benefits, and details about procedures, the more benefits they perceived. Many applicants felt that the self-assessment improved their Read more





0

Philanthropy, Evaluation, Accountability, and Social Change

Key points: Many foundations have substituted process accountability for accountability for contributing to social change. While process accountability is important, it sets a floor, not an aspirational ceiling. There are tools—such as risk analysis, systems approaches, and game theory—that can help philanthropy engage in work on complex social problems that cannot be deconstructed into a series of small, linear projects. Seeking to extend basic human rights to more individuals around the world, seeking to reduce racism in a given city, or seeking to change public health norms in small town—all of these aspirations require first a willingness to take on Read more





0

Philanthropy and Mistakes: An Untapped Resource

Key points: Sharing and leveraging lessons learned from mistakes is an important but underutilized resource to improve philanthropic investments and nonprofit performance. Philanthropic mistakes extend beyond the results of program evaluations to include questions of mission, role, investment strategies, and implementation. Distinguishing between “constructive” and “nonconstructive” mistakes focuses attention on those factors that shape the outcomes for even the most well-designed investments. Sharing and reflecting upon mistakes has the potential to improve philanthropic capacities for anticipation, learning, and adaptation. Philanthropy must recognize the sometimes blurry lines between success and failure, constructive and nonconstructive mistakes, and philanthropic and nonprofit sector accountability.





0

The Blind Men and the Elephant: Learning a Little at a Time About Civic Engagement

Key points: This article, written from the perspective of the evaluator, describes what happened in one community in which four noncollaborating funders were supporting community development programs.  The Treeline Collaborative evolved from grassroots origins to become a leading organization in the community, serving as a one-stop shop for many programs and providing a structure for civic engagement of residents.  A collaborative evaluation would have enabled a deeper understanding of the Treeline Collaborative, the outcomes it attained and missed, and the multiple roles it plays in the community, perhaps leading to more effective program and funding decisions.





0

Youth Civic Engagement for Dialogue and Diversity at the Metropolitan Level

This article analyzes Youth Dialogues on Race and Ethnicity, a foundation-funded program designed to increase dialogue, challenge segregation, and create change in metropolitan Detroit. It draws on multilevel evaluation of the program and analyzes some of the lessons learned.







0

Evaluation With A Diversity Lens: Exploring Its Functions And Utility To Inform Philanthropic Effectiveness

This report reveals practical insights that can help foundations realize greater effectiveness through increasing inclusivity investments. This piece will prove particularly timely and instructive for funders embarking on the practice of evaluation with a diversity lens (EDL). EDL is an approach to program evaluation that emphasizes the importance of incorporating diverse voices (particularly those of intended program beneficiaries) to identify problems and to engage in program design, implementation, and data analysis.