Evaluation Essentials (eBook)

Methods for Conducting Sound Evaluation Research

Beth Osborne Daponte (Autor)

eBook Download: EPUB

2025 | 2. Auflage
356 Seiten
Jossey-Bass (Verlag)
978-1-394-23479-0 (ISBN)

Infos

A nuts-and-bolts introduction to program evaluation for the nonprofit and public sectors

Evaluation Essentials: Methods for Conducting Sound Research, 2nd Edition, is a substantial update to the popular overview of the evaluation process. Virtually every form of externally funded nonprofit activity must be periodically evaluated, and trained evaluators are in higher demand than ever before. This book offers a step-by-step introduction to the process and methods of program evaluation, with over 40 examples from public policy, public health, non-profit management, social work, arts management, education, international assistance, and labor.

Evaluation Essentials covers updates in the field of evaluation, including contribution analyses, as well as current best practices for forming evaluation questions, applying program theory, performing literature reviews, collecting data, creating outcome measures, designing and conducting surveys, grant writing, and much more. This Second Edition also includes an expanded international context, examining evaluation in international organizations. The process outlined in this book is also applicable to policy evaluation and the evaluation of organizational performance, adding additional value to this timely update.

• Gain a foundational knowledge of the principles and practices of program evaluation
• Design evaluations and conduct research, including quantitative and qualitative analysis
• Generate data that can be used to demonstrate a program's impact to funders and stakeholders
• Learn from examples drawn from a broad range of nonprofit organizations

This definitive guide to evaluation will appeal to professionals in fields as diverse as education, policy sciences, public administration, sociology, health, and beyond.

Beth Osborne DaPonte, Ph.D. has taught program evaluation at the graduate level at Yale University's School of Management, Carnegie Mellon University, and the University of Pittsburgh. Additionally, Beth is Principal and Owner of Social Science Consultants, LLC, a firm that provides consulting and advisory services to the non-profit/NGO, religious, governmental, and international sectors. Clients have included the United Nations (secretariat and UN-System funds, programs, and agencies); foundations (e.g., Forbes Funds, Heinz Endowments, Hillman Family Foundations); educational institutions (e.g., University of Pittsburgh); and US-based nonprofits (e.g., Greater Pittsburgh Community Food Bank, Allies for Children).

Textauszug

CHAPTER 1
INTRODUCTION

The role of the program director is to believe, the role of the evaluator is to doubt.

—CAROLE WEISS

LEARNING OBJECTIVES

After reading this chapter, you should be able to

• Explain the primary and secondary goals of the evaluation
• Describe the steps of the evaluation framework

INTRODUCTION

An executive director of a medium‐sized, nonprofit social service agency once told me, “I know we’re doing good—I can see it in our clients’ eyes.” I wish that this had been a unique encounter, but I have repeatedly had people responsible for running programs and delivering services—including board members, executive directors, and staff—say some version of the above. Sometimes, there is variation—“Funders wouldn’t fund us if we weren’t doing good,” or from the funders’ perspective, “They must have done something good since they spent all of the money in the way that they said they were going to.” Potential clients may be influenced by the popularity of a program, thinking that people would only flock to impactful programs.

Unfortunately, none of these sentiments tells us what would have occurred to clients if the program hadn’t existed. Clients may have been just as successful, or even more successful, if left to their own devices or if they were exposed to a different program. Or, perhaps clients who seemed not to have changed were actually better off, since had it not been for the program, they would have been worse off.

In all fairness, if those who deliver services didn’t believe that they are effectively improving people’s lives, they might have difficulty devoting themselves to their jobs. In the nonprofit sector, rather than being primarily motivated by wages and compensation, staff are often motivated by their commitment to their organization’s mission and belief in the effectiveness of the initiative on which they spend their time and resources.

An evaluator’s primary job is to examine rigorously the relevance, impact, efficiency, and sustainability of interventions. Evaluators typically ask questions such as:

• To what extent is the intervention the right way to address the problem it aims to address?
• What is the impact of the intervention on its target population?
• Does the intervention produce unintended consequences?
• To what extent is the intervention being delivered efficiently and how can it be delivered more efficiently?
• Should the intervention be continued and if so, how will it be continued? Who will support it?

By considering these broad questions and interacting with the program designers, current and potential beneficiaries, program governance structures (e.g., boards, trustees, and overseers), and staff, evaluators’ insights can result in program improvements for ongoing programs and improvements in program design for initiatives that have not yet started. Note that this set of questions can be asked retrospectively about interventions (e.g., “was the intervention the right way to address the problem?”) or prospectively (e.g., “to what extent does stakeholder feedback and a literature review indicate that the planned intervention will be the right way to address the problem?”).

An evaluator’s secondary job is to encourage program governance, staff, and planners to be more critical of their existing and future initiatives—encourage them to develop a mindset that, rather than assuming that initiatives are effective, instead asks how to make initiatives more effective and whether an initiative should be continued or (gasp!) abandoned. To design the most impactful programs, boards, planners, and staff need to be more critical of the role that their services (or planned services) play (or will play) in their clients’ lives. What assumptions are programs making about clients? Conversely, what do clients and potential clients assume about programs?

Although clients’ eyes may reflect gratitude, they are not a substitute for rigorous evaluation. For example, they do not reveal whether a program reaches its intended target population or whether clients would have been just as well off without the program. A critical perspective is needed if the efforts are to continually improve over time.

For‐profit organizations have metrics for success that typically involve profit margins, stock prices, and return‐on‐investment. Determining success for nonprofits (in the international sector referred to as nongovernmental organizations or NGOs) is a more nuanced enterprise—success ultimately rests on the degree to which the organization’s mission has been accomplished.

In recent years, funders, management, fundraisers, and potential clients want more information on nonprofits’ impacts and efficiencies. Competition for resources in the nonprofit sector is fierce. With the proliferation of nonprofit organizations, questions about which are the best, which are worth investing in, and which are strong and resilient enough to be sustainable become extremely important.

SIZE AND IMPORTANCE OF NONPROFIT SECTOR

The demand for evaluation services in recent years has exploded, not only because nearly all nonprofits and governmental organizations have realized that they need to critically examine their initiatives, but also because of the growth of the nonprofit sector. Consider that between 2010 and 2020, in the United States, the number of nonprofits (defined for this purpose as an organization that filed a required 990 form with the IRS) increased from 186,417 to 217,263 organizations. In 2020, these organizations held $5.5 trillion in total assets, nearly $2 trillion in total liabilities, and had total revenues of $2.7 trillion. They spent $2.1 trillion on program services.1 To put this in perspective, consider that the USA Gross Domestic Product (GDP) in 2020 was nearly $21 trillion. The nonprofit sector’s spending on program services accounted for approximately 10% of the nation’s GDP.

The nonprofit sector provides services ranging from functions provided by universities, private schools, hospitals, arts and cultural institutions, legal assistance, food assistance, and other safety net and social organizations. Given the magnitude and reach of the nonprofit sector and its importance to the US economy, it is imperative that its entities and funders continue to develop an evaluative mindset. As nonprofits carry out more functions and greater reliance is placed on them, there is increased demand for information about their ability to accomplish their missions efficiently and their impact on the persons, entities, and/or communities they serve.

In recent years, the role of government versus the nonprofit sector has notably changed. The US government is relying more on nonprofits to provide assistance, sometimes with governmental funding and sometimes without. Government increasingly relies on the nonprofit sector, with its army of volunteers and relatively low‐paid staff, to compensate for gaps in governmental programs. This becomes painfully evident during the aftermath of natural disasters, when local food banks and feeding centers run by nonprofits are asked to provide food, the American Red Cross provides shelters, and other nonprofits step up in other ways to serve populations in crisis. But even when there is not an acute emergency, the nonprofit sector is increasingly comprising the safety net for lower‐income persons. Rather than providing services directly, federal, state, and local governments increasingly contract with nonprofits to provide services.

With more reliance placed on the nonprofit sector comes the demand for all nonprofit organizations and especially those that compete with other nonprofits for “market share” to show their effectiveness. Government, consumers, and donors would like to know which nonprofits are most effective at providing comparable services. In this respect, evaluation is viewed as essentially a means of providing information to rank programs and organizations. Evaluation reports are used to answer the questions “which agency should a government contract for a needed service?” and “which nonprofits should foundations and individual donors fund?”

This book is intended to provide leaders, program creators, and program implementers in the nonprofit/NGO/governmental arena with enough knowledge of the evaluation field to empower them to engage with evaluation activities in meaningful ways. Many nonprofit leaders lack training in the nuances of evaluation research. Nonprofit leaders can come to their positions in a variety of ways. For example, the Executive Director of a nonprofit organization that provides guardianship services to the elderly and/or disabled was a noted accountant, but never received any training in evaluation. The same goes for the Executive Director of a local food bank who had been a passionate anti‐hunger advocate and the Executive Director of a local library who excelled as a head librarian.

Many influential foundations lack an evaluation policy that would speak to the size of investments that must be evaluated and the types of evaluation activities that would occur when considering whether to fund an initiative. Some leaders and staff of foundations who have decided that...

Produkt-Details

• Erscheint lt. Verlag: 27.3.2025
• Sprache: englisch
• Themenwelt: Schulbuch / Wörterbuch
• Themenwelt: Sozialwissenschaften ► Pädagogik
• Schlagworte: Evaluation • evaluation book program evaluation book • Evaluation Design • evaluator • evaluator book • grant-funded • nonprofit evaluation • nonprofit fundraising • nonprofit program • Nonprofit Research • Performance Evaluation • Program evaluation
• ISBN-10: 1-394-23479-1 / 1394234791
• ISBN-13: 978-1-394-23479-0 / 9781394234790