Protections to Human Subjects and the Role of Institutional Review Boards
Both the Standards and the Guiding Principles emphasize that to behave ethically, evaluators must protect the rights of people from whom they collect data. Institu- tional Review Boards (IRBs) are committees of five or more peer researchers who review the data collection plans, or protocols, for proposed research and monitor ongoing research to ensure that the rights of human subjects are protected.6 IRBs are governed by the Office of Human Research Protections (OHRP), which is part of the U.S. Department of Health and Human Services. Since 1991, federal regulations
6The words ”human subjects” have historically been used in research to refer to the people who are providing data for the study. That is how we use the words “human subjects” here. However, the word “subjects” implies a helplessness or passivity that many find inappropriate in today’s research and eval- uation endeavors. As do others, we will typically use the word “participants” when referring to the people who provide data for evaluations through completing surveys, participating in focus groups or interviews, permitting observations, etc. We use the words “human subjects” to avoid confusion when citing or discussing work by others who use these words. Thus, for example, IRBs are often called Human Subjects Review Boards.
Chapter 3 • Political, Interpersonal, and Ethical Issues in Evaluation 91
require that organizations that receive federal funds for research have IRBs to re- view all research conducted by the organization. The complete set of regulations is available at http://ohrp.osophs.dhhs.gov. The OHRP can suspend federally funded research if institutions are viewed as noncompliant with regulations. Although such suspensions are “extremely rare and highly controversial,” the threat prompted institutions to review and tighten many IRB procedures (Oakes, 2002, p. 450).
The guidelines for protecting human subjects emerged from the Belmont Report (1979) where the focus was on biomedical research, but the Belmont Report itself was prompted by congressional and public outrage at the infamous, 40-year Tuskegee Syphilis Study.7 Other serious violations in the ethics of social science research have also occurred (Humphreys, 1975; Kahn & Mastroianni, 2001; Milgram, 1974). Insti- tutional Review Boards and their regulations were prompted and guided by the recommendations of the Belmont Report concerning protection of human subjects.
For evaluation studies, the common concern of IRBs is determining whether participants have, in fact, given informed consent to participate in a study. Studies using “vulnerable populations,” typically children, pregnant women or women who may be pregnant, prisoners, and people with limited capacity are given special attention by IRBs and the regulations, at least partly because they may not be able to give full, informed consent. Many evaluation studies, however, may be exempt from IRB review according to the regulations. Specifically, research in educational settings that is intended to study traditional educational practices is exempted from IRB review, as well as data collected through “educational tests,” defined to include surveys, interviews, and observations of public behavior when individuals are not identifiable and data are confidential. However, the individual evaluator should not decide whether his or her study is exempt. Instead, an IRB board should determine if an exempt status is appropriate. This can often be done relatively easily through contact with the IRB or through an expedited review process. In fact, many evalu- ations are reviewed through an expedited review process that involves the research protocols for the study being reviewed by one IRB member.
In recent years, however, IRBs have drawn some criticism for their stringent review of social science research, with critics arguing that some IRB requirements have jeopardized legitimate research. We acknowledge, from personal experience,
7The Tuskegee Syphilis Study or the Tuskegee Experiment was begun in 1932 and continued until it was halted in 1972. The study initially recruited 399 poor African American men who were share- croppers and had syphilis with the purpose of describing the natural progression of the disease. In the 1940s, penicillin was found to be a cure for syphilis and became a common treatment. This informa- tion was withheld from the men in the study and they were left untreated as the study continued for 40 years. The study was halted in 1972 through the efforts of Peter Buxton, a Public Health Service venereal disease investigator. Although his efforts to stop the study began in the late 1960s, he was un- able to stop the study through official channels. He went to the press in the 1970s and congressional hearings were held. Many men died during the course of the study. Forty of their wives were infected and 19 of their children were born with congenital syphilis. In 1997, President Clinton made a formal apology on behalf of the U.S. government, which had funded and conducted the study through the Public Health Service. The study prompted the government to create a commission to write regulations for research, which resulted in the Belmont Report.
92 Part I • Introduction to Evaluation
that individual IRBs do not always provide reasonable feedback and can overstep their boundaries because of lack of knowledge of research and informed consent. Qualitative data collection, where flexibility and adaptation in data collection may be required, can pose particular problems. IRBs may request standardized inter- view questions for review when the evaluator needs the flexibility to adapt ques- tions to the purposes of the evaluation and the previous statements of the person being interviewed. The National Science Foundation has taken a leadership role in trying to clarify guidelines for qualitative research. Their web site containing Frequently Asked Questions (FAQs) about data collection and ethical reviews is particularly informative for readers with these concerns. (See http://www.nsf .gov/bfa/dias/policy/hsfaqs.jsp#exempt.) Oakes’s “Evaluator’s Guide to the IRB” provides more details on the history of IRBs and their requirements (Oakes, 2002).
Our concern in this chapter is making sure that data are collected in a way that protects the rights of participants in an evaluation study. It is not only impor- tant for evaluators to learn the policies of the IRBs that oversee their work and the federal regulations that govern them, but also to consider voluntarily seeking IRB review. We agree with many who study the ethics surrounding human data collection that it is useful for researchers and evaluators to seek the opinions of others about their data collection. Often, the researcher or evaluator is too close to his or her own study to see something that might be a threat. IRBs can provide use- ful input from other researchers who are informed on ethical issues concerning data collection from humans.
Confidentiality and Informed Consent. Confidentiality and informed consent are issues that any evaluator collecting data should be aware of and consider when collecting data. Often, confidentiality and anonymity are confused. Anonymity means that no one knows the identity of the person who provided the data. Confidentiality means that the researcher, evaluator, or person devel- oping the data base may have a code that, in other documents, can be linked to a name, but that the identity of people providing the data will not be revealed to others. Obviously, interviews or observations are not anonymous. The person conducting the interview or observation is aware of the identity of the individ- ual. Similarly, when codes are used on surveys to track who has responded and to prompt those who have not responded to do so, someone is able to make a link among those codes, the responses to a survey, and an individual name. However, the data analysis will not make use of the individual identifiers and, hence, the data are confidential. Further, specific procedures for separating the names and codes from the data and for maintaining the security of the list of names and codes must be established. Any data collection activity should correctly inform in- dividuals as to whether the data they provide should be considered anonymous or confidential.
Informed consent is a central mechanism for protecting the rights of human subjects. As Oakes has written in his Evaluator’s Guide to the IRB, “Informed con- sent is one of the primary ethical requirements underpinning research with human subjects” (2002, p. 463). Informed consent emerged as a central ethical
Chapter 3 • Political, Interpersonal, and Ethical Issues in Evaluation 93
principle after the Nuremberg trials of Nazi scientists who had conducted research on prisoners in concentration camps. The Nuremberg Code, developed after the trials, established the principle that researchers should not collect data from someone without first obtaining their consent and that such consent should be both fully voluntary and informed. “Informed” means that participants should be told of the purpose of the research, its potential risks and benefits to them, the confidentiality of the information and other relevant issues concerning how it will be handled and protected, and what participating in the study will mean for them, that is, the data that will be collected. The voluntary nature of their participation should also be made clear. For example, evaluations are typically conducted in the context of a program. Participants need to know that they can continue receiving the services of the program even if they choose not to partic- ipate in the research. If people receiving services believe they must participate in the research to continue in the program, their participation in the research is not truly voluntary. Informed consent is typically obtained through an informed con- sent form that describes the study, its purpose, the potential risks and benefits, the voluntary nature of participation, how data will be handled, and other rele- vant concerns. But it is important that the language of such forms be clear and understandable to the intended audience. A member of the evaluation team trained in the ethical concerns of informed consent should be present to answer questions a participant may have. (See Fitzpatrick [2005] on informed consent.) IRBs typically pay considerable attention to the issue of informed consent and may have sample consent forms for new evaluators to use as guides for consent forms in their own research.
Cultural Competence and Sensitivity. Finally, ethical data collection involves sensitivity to the cultural norms and beliefs of the individuals and groups from whom one is collecting data. Information that might be considered quite appropri- ate to collect in one group might be considered quite private or misunderstood by another. Such sensitivity is part of attaining cultural competence, as illustrated in the Guiding Principle B.2 and also in D.6. Consider, for example, an evaluation of a school program for children of recently arrived immigrants. Some of those immi- grants may be in the country illegally; however, in most cases, the immigration status of children’s parents would be irrelevant to the evaluation. The evaluator should avoid pressure to collect such data and, more importantly, should consider the wording of individual questions. If the questions appear to be gaining informa- tion that pertains to entry to the country, it may threaten the validity of responses to other items and may not show respect for the privacy of those participating in the evaluation. Evaluators should recognize that evaluation and confidentiality may be suspicious or alien concepts to those completing the survey. Providing information on themselves and their families may be a threatening or frightening experience. Needless to say, surveys should be translated into a language and words that parents can read and understand. Interviews should be conducted by individuals who are not only fluent in the language, but have good knowledge of the culture and norms of the particular immigrant group.
94 Part I • Introduction to Evaluation
In closing, let us emphasize that evaluators have an ethical obligation to con- sider the rights of those from whom they are collecting data and to make sure that those rights are protected. Seeking input from others—whether it be from an IRB committee or other informed researchers, members of an advisory group, clients, or representatives of the individuals from whom data are collected—should be a central part of that process. Individuals providing data should be informed of the purposes of the research and any risks that may be incurred by participating. Further, the evaluator should collect only the data necessary and essential for the evaluation. An evaluation does not give one license to collect irrelevant data or to unnecessarily intrude on the privacy of individuals.
Learning and Practicing Ethical Behavior. In this section, we have attempted to acquaint the reader with the standards and codes that have been developed to guide evaluators. But applying these standards or codes to individual evaluations is a much different issue. As the Joint Committee emphasizes, not all standards are equally important in every evaluation. Choices must be made. Similarly, while the Guiding Principles are intended to “proactively guide the behaviors of profes- sionals in everyday practice” (American Evaluation Association, 2004, Preface C), deciding how to apply them in a specific evaluation, particularly when conflicts emerge, requires careful consideration and tough choices. Morris and Cooksy have helped make us aware of the complexity of these choices through an ongo- ing column on ethical dilemmas in the American Journal of Evaluation. The column, begun by Morris in 1998 and assumed by Cooksy in 2004, presents an ethical problem and calls upon two different, experienced evaluators to describe how they would respond to the issue. Distinct differences emerge. See, for example, the disagreements between Cooksy and Knott on an ethical problem concerning sexual harassment by a manager that was reported during confidential interviews (Morris, Cooksy, & Knott, 2000). These differences help educate and sensitize evaluators to recognize and analyze ethical problems they encounter and consider the choices they make. We encourage readers to read some of these ethical dilem- mas and the responses to consider how they would respond and to refine their skills in ethical reasoning for evaluation practice.