Educational Connoisseurship and Criticism
In the previous section, we discussed applications of the expertise-oriented approach in which the experts are not necessarily evaluators. They are experts in something else—the content they are judging. Further, these applications are examples of the expertise-oriented approach, but they were formed and exist independent of the professional evaluation community. In other words, we can study these processes as examples of expertise-oriented evaluation approaches, but those in the evalu- ation community are generally not involved in establishing these activities or in conducting them, as is the case with the other approaches we will discuss. As noted, we have begun our discussion of approaches by focusing on the oldest eval- uation approach, one used for centuries before formal program evaluation emerged, to make judgments about important issues.
But, the expertise-oriented approach has also been part of the discussion of evaluation theories. In the early days of evaluation, Elliot Eisner was a key figure in discussing what evaluation should be, and his writings provide the theoretical foundation for the expertise-oriented approach and connect it to the evaluation literature (Eisner, 1976, 1985, 1991a, 1991b, 2004). Alkin and Christie (2004), in their evaluation tree depicting the origins and theories of evaluation, place Eisner, along with Michael Scriven, at the base of the valuing branch because their em- phasis was on the valuing role of evaluation—determining the value, the merit or
140 Part II • Alternative Approaches to Program Evaluation
worth, of the thing being evaluated. Eisner drew from the arts to describe his ap- proach to evaluation. His perspective was a useful counterpoint to the emphasis in the 1970s on social science methods and program objectives. We will briefly discuss his concepts of connoisseurship and criticism, the fundamentals of his eval- uation approach. These concepts fall within the expertise-oriented approach, because they require expertise in identifying and judging critical components or elements of the thing being evaluated.
The roles of the theater critic, art critic, and literary critic are well known and, in the eyes of many, useful roles. Critics are not without their faults. We may disagree with their views, but their reviews are good examples of direct and efficient application of expertise to that which is judged. Their criticism prompts us to think about the object being evaluated in different ways, even if we continue to disagree with their judgment. That is one goal of a written review or criticism: To prompt us to think about elements of the object that we, as nonexperts, might not have considered. Eisner (1991a) proposes that experts, like critics of the arts, bring their expertise to bear in evaluating the quality of programs in their areas of proficiency. Eisner does not propose a scientific paradigm but rather an artistic one, which he sees as an important qualitative, humanistic, nonscientific supplement to more traditional inquiry methods. He argues that we need to see the thing being evalu- ated from multiple perspectives and that the emphasis on quantitative, reduction- ist methods fails to convey many important qualities of the whole. He notes that numbers play a role in educational evaluation, his area of interest, but also limit what we see:
[W]e should be recognizing the constraints and affordances of any form of rep- resentation we elect to use. Just as a way of seeing is also a way of not seeing, a way of describing is also a way of not describing. The tools we employ for notic- ing have an enormous impact on what it is that we become aware of. If we want a replete, fulsome, generous, complex picture of a classroom, a teacher, or a student, we need approaches to that perception of such phenomena and, in addition, a form of presentation that will make those features vivid. (Eisner, 2004, p. 200)
The key elements of Eisner’s approach are connoisseurship and criticism (Eisner, 1975, 1991b). Connoisseurship is the art of appreciation—not necessar- ily a liking or preference for that which is observed, but rather an ability to notice, “to recognize differences that are subtle but significant in a particular qualitative display” (Eisner, 2004, p. 200). The connoisseur has developed knowledge of the important qualities of the object and the ability to observe and notice them well and to study the relationships among them. The connoisseur, in Eisner’s view, is aware of the complexities that exist in observing something in real-world settings and possesses refined perceptual capabilities that make the appreciation of such complexity possible. The connoisseur’s perceptual acuity results largely from a knowledge of what to look for (advance organizers or crit- ical guideposts) gained through extensive previous experience, education, and reflection on that experience.
Chapter 5 • First Approaches: Expertise and Consumer-Oriented Approaches 141
The analogy of wine tasting is used by Eisner (1975) to show how one must have many experiences to be able to distinguish what is significant about a wine, using a set of techniques to discern qualities such as body, color, bite, bouquet, flavor, and aftertaste, to judge its overall quality. The connoisseur’s refined palate and gustatory memory of other wines tasted is what enables him or her to distinguish subtle qualities lost on an ordinary drinker of wine and to render judgments rather than mere preferences. Connoisseurs exist in all realms of life, not solely the gusta- tory or artistic. Eisner describes a good coach as a connoisseur of the game who, when watching others at the sport, can recognize subtleties that those with less experience would miss: “We see it displayed in blazing glory in watching a first-rate basketball coach analyze the strengths of the opponents, their weaknesses, as well as the strengths and weaknesses of the team that he or she is coaching” (2004, p. 198).
Connoisseurship does not, however, require a public description or judgment of that which is perceived. The public description is the second part of the Eisner approach. “Criticism,” Eisner states, “is the art of disclosing the qualities of events or objects that connoisseurship perceives” (1979a, p. 197), as when the wine con- noisseur either returns the wine or leans back with satisfaction to declare it of acceptable, or better, quality. Or, more akin to public evaluation, criticism is when the wine critic writes a review of the wine. Evaluators are cast as critics whose con- noisseurship enables them to give a public rendering of the quality and significance of that which is evaluated. Criticism is not a negative appraisal but rather an educational process intended to enable individuals to recognize qualities and char- acteristics that might otherwise have been unnoticed and unappreciated. Criticism, to be complete, requires description, interpretation, and evaluation of that which is observed. “Critics are people who talk in special ways about what they encounter. In educational settings, criticism is the public side of connoisseurship” (Eisner, 1975, p. 13). Program evaluation, then, becomes program criticism. The evaluator is the instrument, and the data collecting, analyzing, and judging are largely hidden within the evaluator’s mind, analogous to the evaluative processes of art criticism or wine tasting. As a consequence, the expertise—training, experience, and credentials—of the evaluator is crucial, because the validity of the evaluation de- pends on the evaluator’s perception. Yet different judgments from different critics are tolerable, and even desirable, since the purpose of criticism is to expand per- ceptions, not to consolidate all judgments into a single definitive statement.
Eisner’s educational criticism focuses on four dimensions that should be portrayed in a criticism: description, development of themes, interpretation, and evaluation. The focus is on expert, and sometimes, detailed description of the factors that are important in judging the quality of the product or program. Obviously, the approach would not be the most direct for clearly establishing cause-and-effect relationships, but it can be useful in helping us to understand the nature of the in- tervention and the manner in which it leads to different outcomes. As Eisner recently stated, “Educational connoisseurship and educational criticism represent an effort to employ what the arts and humanities as partners with the social sciences have to of- fer in advancing our understanding of the process and effect of education. In an age of high-stakes testing, it is a perspective we badly need” (Eisner, 2004, p. 202).