Evaluation and Its Political Context
Was it mere naïveté that accounted for the initial failure of evaluation researchers to anticipate the complexities of social and political reality? These researchers [evaluators] were mentally prepared by the dominant Newtonian paradigm of social science for a bold exploration of the icy [unchanging] depths of interplane- tary space. Instead, they found themselves completely unprepared for the tropical nightmare of a Darwinian jungle: A steaming green Hell, where everything is alive and keenly aware of you, most things are venomous or poisonous or otherwise dangerous, and nothing waits passively to be acted upon by an external force. This complex world is viciously competitive and strategically unpredictable because [evaluation] information is power, and power confers competitive advantage. The Darwinian jungle manipulates and deceives the unwary wanderer into serving myriads of contrary and conflicting ends. The sweltering space suits just had to come off. (Sechrest & Figueredo, 1993, p. 648)
66 Part I • Introduction to Evaluation
This colorful portrayal of evaluators’ first forays into the complex and unpre- dictable environment in which programs are managed and evaluated underscores a critical point: Evaluators work in a political environment. Evaluation itself is a political act—and the professional evaluator who prefers to eschew “politics” and deal only with technical considerations has made a wrong career choice.
From the beginning of modern-day evaluation, evaluators have written about the political nature of the activity. Suchman (1967), Weiss (1973), and Cronbach and his colleagues (1980) all emphasized the political nature of eval- uation, underscoring the fact that evaluation of publicly supported enterprises is inextricably intertwined with public policy formulation and all the political forces involved in that process. However, as Sechrest and Figueredo’s descrip- tion at the beginning of the chapter so vividly indicates, researchers moving into the political arena to conduct evaluations during its time of growth in the United States in the 1970s were unaware of the implications that working in a political environment had for their methodological work. (See also Datta and Miller [2004], and Weiss and Mark [2006] for their descriptions of these early evaluations.)
Today, at least partly because the field has had time to mature and gain more experience in conducting evaluations and to consider the factors that influence their success, evaluators are much more aware that they work in a po- litical environment. Nevertheless, perhaps because the training of evaluators tends to emphasize methodology, evaluators continue to be surprised at the political context of their work and a little unsure of what to do in it (Chelimsky 2008; Leviton 2001). Another explanation for at least U.S. evaluators’ naïveté about the political world may rest with the disciplines they studied. A study of members of the American Evaluation Association found that the most common fields of study for U.S. evaluators were education and psychology (American Evaluation Association, 2008). Unlike European evaluators (Toulemonde, 2009), few evaluators in the United States were trained in the fields of political science or economics and, therefore, consideration of politics and the political context may be relatively new to them. Shadish, a leader in evaluation theory and methodology who was trained in psychology, remarks on his coming to un- derstand that politics played an important role in evaluation (Shadish & Miller, 2003). He tells of his surprise years ago that people did not choose to adopt a program that had been proven to be quite successful. The occasion prompted him to read and then write an article on policymaking in the American Psycholo- gist, the leading psychology journal (Shadish, 1984). He notes that in preparing the article he read “Politics and Markets” by Charles Lindblom, an esteemed political scientist, along with some other important works in economics and political science, and “all of a sudden I realized that the world didn’t work around what was effective. It worked on other matters entirely—on politics and economics” (Shadish & Miller, 2003, p. 270).
In this section, we will discuss the reasons why evaluation is political and the nature of that political environment. Then, we will provide a few suggestions for how evaluators can work effectively in a political world.