Philosophical and Ideological Differences
Logical Positivism. Early evaluations emerged from the social sciences, in particular education and psychology, at a time when the dominant paradigm was positivism. Logical positivists, a more extreme branch of positivism, argued that knowledge was obtained entirely through experience, specifically through observation, and held rigid views concerning the world and data collection (Godfrey-Smith, 2003). They argued that (a) there is one reality of the objects we are studying and the aim
Chapter 4 • Alternative Views of Evaluation 115
of researchers and evaluators is to use social science research methods and theories of statistical probability to discover that one reality and to establish laws and theo- ries about how things work, and (b) to effectively gain knowledge of that reality, researchers need to be “scientifically objective.” A key component of that approach is that researchers should maintain some distance from the program to be studied so as not to influence the program itself, the participants, or the results of the study. The methods used to achieve this objectivity, or distance, were typically quantita- tive in nature. Objectivity or objectivism, meaning that the researcher’s views and values do not influence the results obtained, was a key principle of positivism.
Postpositivism. Reichardt and Rallis (1994) note that logical positivism began to decline around the time of World War II, though elements of positivism continued to influence research and evaluation for some time. By 1984, however, Donald Campbell, a prominent research methodologist and evaluator with a quantitative orientation, noted that “twenty years ago logical positivism dominated the philos- ophy of science. . . . Today the tide has completely turned among the theorists of science in philosophy, sociology, and elsewhere. Logical positivism is almost uni- versally rejected” (p. 27). Postpositivism emerged in reaction to logical positivism and many, unfortunately, confuse the two. Guba and Lincoln (1989) argued that the views of postpositivists were not compatible with other approaches to evalua- tion. However, Reichardt and Rallis (1994), quantitative and qualitative evaluators respectively, effectively refuted their arguments, demonstrating that postpositivists, such as Campbell and Stanley (1966) and Cook and Campbell (1979), did not hold the views of logical positivists. Instead, they showed through quotations from their work that these postpositivists, and others, recognized that facts and methods or inquiry choices in research are influenced by the values of the researcher, that knowledge is fallible and changing, that data can be explained by many different theories, and that reality is constructed by people and their experiences.
The focus of postpositivists, however, was on examining causal relationships to develop laws and theories to describe the external world, albeit temporary ones given the fallibility of knowledge. Replication and intersubjectivity, not objectiv- ity, were the keys to ensuring good research (Frankfort-Nachmias & Nachmias, 2008). Intersubjectivity involves the ability to communicate what one does in research in such a way that others can judge its findings and replicate it to see if they obtain the same results. For evaluation, House and Howe (1999) note that one of the key characteristics of this philosophical approach, which they call the received view, is viewing facts as quite distinct from values and believing that eval- uators should be focusing on the facts.
A Constructivist Paradigm. As evaluation continued, evaluators saw that con- text and values played very important roles in evaluation. Unlike many laws of science which are readily generalizable from one setting to the next, the factors that influence the success of education, social, and economic programs can differ dramatically from one setting to another. Also, clients and stakeholders for the
116 Part II • Alternative Approaches to Program Evaluation
evaluation often had information needs that were not so concerned with estab- lishing causality as with gaining a better understanding of the program and those they served. Program developers recognized the many differing “realities” or con- ditions or life experiences of those that the programs were intended to serve and saw that programs had different effects on different kinds of clients. They wanted to know more about these issues to help them improve their programs. And val- ues were an integral part of what programs, policies, and evaluations confronted. To exempt evaluation from such values was to make it incomplete.
The constructivist paradigm that was emerging then corresponded more closely to the views and experiences of these evaluators and program developers. Constructivists took a different view of ontology and epistemology (Guba & Lincoln, 1994). Although we now realize that the differences were not as extreme as they were sometimes portrayed, Guba and Lincoln focused on understanding our constructed world and, in particular, the multiple realities seen or experienced by different stakeholders. They argued that objectivity was not possible; we each see the world through our own lens, influenced by our own experiences. Later, House and Howe (1999) emphasized that the fact-value dichotomy, or the rigid distinc- tion between “facts” which are objective and “values” which are subjective, is in fact (pun intended) a continuum. Our values influence what we perceive to be facts. Thus, evaluators should become involved with values—helping stakehold- ers articulate their values, considering the values inherent in the evaluation, and working to portray the program through different stakeholders’ perspectives of reality. Constructivism also continued its focus on what Schwandt (1997) calls the “localness” of knowledge. Evaluation is intended to provide understanding of a particular program and its context and is less concerned with generalizability and developing laws and theories for other settings.