Which do you prefer, an external or internal evaluator?
Major Concepts and Theories
1. Evaluation is the identification, clarification, and application of defensible criteria to determine an evaluation object’s value, its merit or worth, in regard to those criteria. The specification and use of explicit criteria distinguish formal evaluation from the informal evaluations most of us make daily.
2. Evaluation differs from research in its purpose, the role of the evaluator and the researcher in determining the focus of the study, the criteria used to judge its quality, its involvement of stakeholders, and the competencies required of those who practice it.
3. The basic purpose of evaluation is to render judgments about the value of the object under evaluation. Other purposes include providing information for program and organizational improvement and to make decisions, working to better society and to improve and sustain democratic values, encouraging meaningful dialogue among many diverse stakeholders, as well as adding to our knowledge concerning the application of social science theory, and providing oversight and compliance for programs.
4. Evaluators play many roles including facilitator, planner, advocate, scientific expert, critical friend, collaborator, and aid to decision makers and other stakeholder groups.
5. Evaluations can serve formative or summative decisions as well as other purposes. Formative evaluations are designed for program improvement. The audience is, most typically, stakeholders close to the program. Summative evaluations serve decisions about program adoption, continuation, or expansion. Audiences for these evaluations must have the ability to make such “go-no go” decisions.
6. Evaluations can address needs assessment, process, or outcome questions. Any of these types of questions can serve formative or summative purposes.
7. Evaluators may be internal or external to the organization. Internal evaluators know the organizational environment and can facilitate communication and use of re- sults. External evaluators can provide more credibility in high-profile evaluations and bring a fresh perspective and different skills to the evaluation.
8. Evaluation goes beyond particular methods and tools to include a way of thinking. Evaluators have a role in educating stakeholders and the public about the concept of evaluation as a way of thinking and reasoning. This way of thinking includes acknowl- edging, valuing, using, and exploring different perspectives and ways of knowing, and creating and encouraging an experimenting society—one that actively questions, con- siders, and creates policies, programs, interventions, and ideas.
36 Part I • Introduction to Evaluation
Discussion Questions
1. Consider a program in your organization. If it were to be evaluated, what might be the purpose of the evaluation at this point in time? Consider the stage of the pro- gram and the information needs of different stakeholder groups. What might be the role of evaluators in conducting the evaluation?
2. What kind of evaluation do you think is most useful—formative or summative? What kind of evaluation would be most useful to you in your work? To your school board or elected officials?
3. Which do you prefer, an external or internal evaluator? Why?
4. Describe a situation in which an internal evaluator would be more appropriate than an external evaluator. What is the rationale for your choice? Now describe a situation in which an external evaluator would be more appropriate.
Application Exercises
1. List the types of evaluation studies that have been conducted in an institution or agency of your acquaintance, noting in each instance whether the evaluator was internal or external to that institution. Determine whether each study was form- ative or summative and whether it was focused on needs assessment, process, or outcome questions. Did the evaluation address the appropriate questions? If not, what other types of questions or purposes might it have addressed?
2. Think back to any formal evaluation study you have seen conducted (or if you have never seen one conducted, find a written evaluation report of one). Identify three things that make it different from informal evaluations. Then list ten informal eval- uations you have performed so far today. (Oh, yes you have!)
3. Discuss the potential and limitations of program evaluation. Identify some things evaluation can and cannot do for programs in your field.
4. Within your own organization (if you are a university student, you might choose your university), identify several evaluation objects that you believe would be ap- propriate for study. For each, identify (a) the stakeholder groups and purposes the evaluation study would serve, and (b) the types of questions the evaluation might address.
Chapter 1 • Evaluation’s Basic Purpose, Uses, and Conceptual Distinctions 37
Case Studies
In this edition, we begin a new practice to ac- quaint readers with real evaluations in order to give them a better understanding of the prac- tice of evaluation. At the end of many chapters, we will recommend one or more interviews that Jody Fitzpatrick, one of our authors, or Christina Christie conducted with a well- known evaluator concerning one evaluation he or she completed. Each article begins with a brief summary of the evaluation. Fitzpatrick or Christie then interviews the evaluator about the choices he or she made in determining the purposes of the evaluation, involving stake- holders, selecting designs and data collection methods, collecting the data, reporting the re- sults, and facilitating use. Interested readers may refer to the book that collects and analyzes these interviews: