Neutrality, Transparency, and Purpose in Accreditation.
Other criticisms of the current approach concern reviewers’ neutrality or objectivity and the trans- parency of the process. Evaluations are expected to be based on independent judg- ments. Such independence is intended to lead to more objective, and hence more valid, judgments of quality. Generally speaking, expertise-oriented evaluators should not be closely affiliated with the institution or product they are judging. For example, we are suspicious of an expert’s endorsement of a product when we know the expert has a financial relationship with the product’s manufacturer. Consider, for example, current discussions of the objectivity of medical research on the effectiveness of a drug when the research is funded by the pharmaceutical company that developed the drug. But accreditation processes make use of peer reviewers who are faculty and administrators from higher education institutions in the region. Accrediting organizations argue that these experts are in the best po- sition to make the judgments and provide the advice institutions need, because they know what can be accomplished in the environment of such an institution—and how to accomplish it. They have worked in it themselves. Critics, however, are concerned that the closeness of the experts to those being judged and possible competition between institutions or departments present serious conflicts of in- terest that can lead to biased judgments. Judgments as blunt as Flexner’s evalua- tions of medical schools would not see the light of day, at least in written reports.
Concerns over objectivity are heightened by the lack of transparency in the process. The U.S. Department of Education would like data and reports to be far more open, meaning that they would be available to parents, students, and the public and would contain content that is readily understood by nonexperts. For example, the Spellings Commission advocated tables presenting data on the
136 Part II • Alternative Approaches to Program Evaluation
knowledge and skills of graduates and other outcome measures for various colleges and universities. These tables would be available for the public to use in judging the quality of institutions, and for other colleges to use as benchmarks (U.S. Department of Education, 2006). Accreditors rely on the thick descriptions contained in self-study reports and the accreditation report. Defenders of the cur- rent system agree that the system relies heavily on confidentiality but argue that this confidentiality is one of the reasons for its success. Because of it, “institutions can be candid in their self-studies, and teams can be honest in their assessments” (O’Brien, 2009, p. 2). If reports were made public, those writing the self-report would be reluctant to discuss real problems, and accreditation teams would edit their wording for public consumption. Neither would facilitate learning about problems and making recommendations for change.