The Expertise-Oriented Approach
The expertise-oriented approach to evaluation is probably the oldest type of formal, public evaluation and, as its name implies, it relies primarily on professional expertise to judge the quality of an institution, program, product, or activity. For example, the merits of a leadership training program for school principals could be assessed by experts from various fields including leadership, educational adminis- tration, and training who would observe the program in action, examine its mate- rials and underlying theory, perhaps interview some trainers and participants, or, in other ways, glean sufficient information to render a considered judgment about its value.
In another case, the quality of a hospital could be assessed by looking at its spe- cial programs, its operating facilities, its emergency room operations, its in-patient operations, its pharmacy, and so on, by experts in medicine, health services, and hospital administration. They could examine facilities and equipment/supplies of the hospital, its operational procedures on paper and in action, data on the fre- quency and outcomes of different procedures, the qualifications of its personnel, patient records, and other aspects of the hospital to determine whether it is meeting appropriate professional standards.
Although professional judgments are involved to some degree in all evalua- tion approaches, this one is decidedly different from others because of its direct, open reliance on professional expertise as the primary evaluation strategy. Such expertise may be provided by an evaluator or by subject-matter experts, depend- ing on who might offer most in the substance or procedures being evaluated. Usually one person will not own all of the requisite knowledge needed to adequately evaluate the program, institution, or agency. A team of experts who complement each other are much more likely to produce a sound evaluation.
Several specific evaluation processes are variants of this approach, including doctoral examinations administered by a committee, proposal review panels, site visits and conclusions drawn by professional accreditation associations, reviews of institutions or individuals by state licensing agencies, reviews of staff performance for decisions concerning promotion or tenure, peer reviews of articles submitted to professional journals, site visits of educational programs conducted at the behest of the program’s sponsor, reviews and recommendations by prestigious blue-ribbon
128 Part II • Alternative Approaches to Program Evaluation
TABLE 5.1 Some Features of Four Types of Expertise-Oriented Evaluation Approaches
Type of Expertise-Oriented Evaluation Approach
Existing Structure
Published Standards
Specified Schedule
Opinions of Multiple Experts
Status Affected by Results
Formal review system Yes Yes Yes Yes Usually
Informal review system Yes Rarely Sometimes Yes Usually
Ad hoc panel review No No No Yes Sometimes
Ad hoc individual review No No No No Sometimes
panels, and even the critique offered by the ubiquitous expert who serves in a watchdog role.
To impose some order on the variety of expertise-oriented evaluation activ- ities, we have organized and will discuss these manifestations in four categories: (1) formal professional review systems, (2) informal professional review systems, (3) ad hoc panel reviews, and (4) ad hoc individual reviews.