Abstract
MONITORING OF RESEARCH BY RESEARCH ETHICS BOARDS HAS BEEN RECOMMENDED by various organizations that fund clinical studies and by other groups. However, little evidence has been reported on the processes, costs and outcomes of these activities, information that would be helpful to guide the boards in their current work and future policies. We report here 3 years of monitoring experience by the research ethics board of a 313-bed university-affiliated community hospital. Activities newly implemented at the beginning of the study period included the use of recruitment logs, audits of completed consent forms and interviews with research subjects. Over the study period, we monitored 33 protocols, through 188 consent form audits and interviews with 17 research subjects. In addition, 26 of 34 research investigators and collaborators responded to a survey about the monitoring. In general, the investigators were supportive of monitoring activities, but most were not willing to contribute financially. The types of monitoring we conducted are feasible and may be suitable (or could be adapted) for use in other institutions.
Monitoring of clinical research enables research ethics boards to ensure that the standards that they approve in theory are being applied in practice.1,2 It has been suggested that such review be performed regularly (e.g., annually) and that it include monitoring of a variety of aspects of research, such as the consent process, adherence to the approved protocol and data integrity.1,3 Although various bodies have recommended that research be monitored as it progresses, a study over the period 1990 to 1993 indicated that few research ethics boards affiliated with Canadian faculties of medicine monitored projects after they had been approved.4
In 1998 the 3 Canadian government research funding bodies issued the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans,5 which requires each institution conducting funded research to establish a monitoring program. The policy statement suggests that, in addition to the minimum requirement of submission of an annual status report to the research ethics board, review of research projects exceeding the threshold of minimal risk might include the following aspects: formal review of the informed consent process, establishment of a committee to monitor safety, periodic review by a third party of the documents generated by the study, review of reports of adverse events, review of patients' charts and a random audit of the informed consent process.5 The purposes of monitoring include education of research staff, quality assurance and prevention of research misconduct.
In spite of increasing requirements for research ethics boards to monitor research, little evidence has been reported on the processes, costs and outcomes of these activities. In this paper, we report on 3 years of experience of monitoring research at a community hospital.
Monitoring at St. Mary's Hospital Centre
St. Mary's Hospital Centre, Montreal, is a 313-bed university-affiliated community hospital without a research institute. Review of research activities at the hospital is coordinated by the Research Review Office in the Department of Clinical Epidemiology and Community Studies, following a set of policies developed in 1994, within the framework of the research policies of McGill University. Hospital research policies are set by the Scientific Research Committee, and scientific and ethics review is conducted by the hospital's Research Ethics Committee. Both groups report to the hospital's board through the vice-president for professional services. The Research Ethics Committee meets monthly for about 2 hours. Each year, on average, the committee approves 22 new protocols and conducts annual reviews on 46 protocols.
Before 1997, monitoring of approved research protocols consisted only of annual reviews and reporting of adverse events. In 1997, after review and discussion of the paper by Weijer and colleagues,1 the hospital expanded its program of research monitoring. The expanded program was approved by the Scientific Research Committee, which granted up to $5000 per year in support of these activities from the research overhead received from nongovernmental research funds (20% of the direct costs of the research). The work has been conducted by several part-time monitoring assistants, who also work as research assistants on other hospital projects and who report regularly to the Research Ethics Committee. Review of the monitoring activities is now a regular part of the committee's agenda.
Monitoring policies and activities
We describe here the monitoring policies and activities put into place in 1997.
Before a new research protocol is approved, investigators are asked to describe the characteristics of patients who would be eligible for the protocol, and an informal check is done to determine whether there is overlap with the patient populations of protocols already under way. On initial approval of a protocol, the Research Ethics Committee specifies the duration of approval (usually 1 year), sets any other conditions (e.g., exclusion of subjects who have consented to participate in other protocols) and decides what type of monitoring is appropriate, taking into account factors such as the type of protocol (e.g., whether it poses more than minimal risk) and the experience of the investigator. (In this paper, the term investigator is used to refer to the person, usually a staff member of St. Mary's Hospital Centre, who is considered by the Research Ethics Committee to be the primary contact person responsible for the protocol; this person's actual role on the research protocol may be principal investigator, co-investigator or local collaborator.) These decisions are communicated in writing to the investigator. A monitoring assistant then contacts the investigator, or a research assistant if the investigator cannot be reached, to explain these requirements in more detail.
Investigators are asked to submit, on a regular basis, a log of subjects recruited to their research protocols, including hospital chart numbers if applicable. The latter are entered into a database and checked to determine whether any of the subjects are participating in more than one research protocol.
Consent form audits are requested routinely for protocols that involve written consent. Unless a waiver has been granted by the Research Ethics Committee, any consent form for hospital research must be filed in the patient's medical record, with a cover sheet briefly summarizing the research interventions, outlining any potential risks and including the name and telephone number of one of the investigators. The medical charts for a random sample of research participants are checked periodically to ensure that the documentation is being filed as required. For research protocols that do not involve hospital patients, but for which the Research Ethics Committee has undertaken responsibility, consent forms are reviewed in the investigator's office. For all consent form audits, the monitoring assistant determines that the consent form is the one approved by the Research Ethics Committee and checks the signatures and dates for completeness and consistency. The investigators and the Research Ethics Committee are informed of the results of consent form audits, and investigators are asked to explain, in writing, any discrepancies.
Interviews with research subjects may be requested for protocols involving more than minimal risk. The protocol is first reviewed to determine an appropriate time to contact the subjects and the types of questions to be asked. A meeting is held with the investigator to arrange a method of contacting subjects. Subjects are usually selected sequentially from the subject log provided by the investigator, with certain exclusions suggested by the investigator (e.g., subjects considered to be too ill to participate). A letter from the chair of the Research Ethics Committee, explaining the purpose of the interview and the voluntary nature of participation, is sent to the subjects or given to them by the investigator. The monitoring assistant then contacts the subjects, either by telephone or in person, and requests verbal consent to conduct the interview. The interview itself is semistructured; it probes the subjects' understanding of the research in which they are participating but does not test them on specific points.
The Research Ethics Committee has the option of requesting other types of monitoring, but none of these have been used so far. These other methods include monitoring of adherence to approved protocols, monitoring of the integrity of data and presence of an observer during consent procedures. The Research Ethics Committee requests from each investigator an appropriate plan to assure and evaluate the quality of protocol data over the duration of data collection, but does not itself monitor this aspect of protocols.
Survey to evaluate monitoring
In February 2000 an anonymous evaluation questionnaire was sent to the 34 investigators whose protocols had been designated for some type of monitoring. The investigators used a Likert-type response scale to give their opinions on 6 statements. They were also asked for comments on the hospital's monitoring program.
Results
Between January 1997 and March 2000, 67 research protocols were approved by the St. Mary's Research Ethics Committee (either initial approval or reapproval). No monitoring was required for 9 of the protocols, either because written consent forms were not used or because the study was being performed at another site. The investigators for the other 58 protocols were asked to provide recruitment logs. Consent form audits were conducted for 33 of these protocols, and a selection of research subjects were interviewed for 6 of these. There were a variety of reasons why consent forms for the other 25 protocols were not reviewed: protocol had not been implemented,5 no subjects had been enrolled so far,5 recruitment log had not yet been received from the investigator,7 the number of subjects enrolled so far was insufficient for a meaningful review of consent forms,3 or a conflict of interest existed because the monitoring assistant was also a research assistant on the protocol.5
Monitoring of recruitment logs
Several instances were identified in which the same individual was participating in more than one research protocol. In most cases, these were nonconflicting situations, for example, no overlap in the time frame for the protocols or minimal burden on the research subject. For example, in one situation, the same patient was enrolled in 3 different but related protocols; however, the Research Ethics Committee concluded that this did not pose an undue burden on the patient or the caregiver. In another situation, 2 protocols were approved for the same population, with the second protocol having a limited period for recruitment. The Research Ethics Committee helped to negotiate an agreement between the investigators to allow both to achieve their enrolment targets, while preventing the same patients from being approached for more than one protocol.
Consent form audits
Consent form audits were conducted for 33 protocols. From the total of 1390 research subjects enrolled in these protocols, a sample of 188 (13.5%) was selected for review of consent forms (Table 1). Required forms were missing or incomplete for a substantial proportion of the 123 hospital charts audited, but there were few problems with missing or incomplete documentation for the 65 office charts (Table 1). Of the 158 consent forms available for review, 6 (3.8%) were different from the one approved. For 2 subjects (1.3%), both enrolled in the same protocol, there was a discrepancy between their age and the age criteria specified in the protocol. This turned out to be a legitimate change in the protocol, approved by the funding organization, that had not been conveyed by the investigator to the Research Ethics Committee. Four subjects (2.5%), who were participating in 1 of 2 protocols conducted by the same investigator, had been enrolled, on the basis of verbal consent only, before the date on which the Research Ethics Committee granted approval for the protocol. Discrepancies between the date on which the subject signed the consent form and the enrolment date given on either the recruitment log or the cover sheet were found in 11 cases (7.0%), and for 3 consent forms (1.9%), the subject signatures were undated. One consent form returned by mail had not been signed by the subject. On another consent form, the letter X appeared in place of the subject's signature, but no explanatory note was provided; it was later confirmed that the subject was unable to sign. Two forms without the subject's signature were accompanied by notes stating that verbal consent had been given. Witness signatures were missing from 13 consent forms (8.2%), 4 of which had been returned by mail, and investigator signatures were missing from 11 (7.0%). Other discrepancies (1 case each) included witness signature different from the printed name, signature date for the witness different from signature date for the subject, incorrect investigator name and undated investigator signature.
Interviews with research subjects
A total of 17 interviews were conducted in connection with 6 drug trials, 13 with the subjects and 4 with family members of incompetent subjects. An additional 2 subjects refused to participate, and another subject could not be contacted.
One subject did not know enough English or French to be interviewed in either of these languages, although this person had completed the consent form in one of these languages. This situation raised the question of whether the subject had been capable of providing informed consent. Of the remaining 16 subjects, 3 (19%) had little understanding of experimental treatments and procedures and those that were part of usual care; they also had little understanding of the possible risks and benefits of participating in the study. Six (38%) of the subjects mentioned that a particular factor had influenced their decision to participate: 1 subject had felt pressured because of the limited time available for making a decision, 2 subjects had been influenced by the enthusiasm of their physician (who was also the investigator), 2 subjects reported that a health care professional (the nurse administering the consent form or the subject's physician) had recommended participation, and 1 subject decided to participate because the study medication was free, whereas payment would have been required for usual treatment. When asked for other comments on their research experience, 3 subjects (19%) mentioned that the investigator had not been approachable; the same investigator was identified in all 3 cases.
Survey of investigators
Of the 34 questionnaires sent to investigators, 26 (76%) were returned. Table 2 shows that most respondents found the monitoring assistant helpful and courteous; however, 3 investigators commented that they had never met the monitoring assistant, and 2 stated either that they were not aware of the monitoring requirements or that the monitoring assistant had met with his or her research assistant only. Most respondents considered the monitoring activities important and useful but did not agree that research grants should include monitoring costs.
Interpretation
We have described the process and some results of one hospital's effort to monitor research protocols approved by its Research Ethics Committee. These activities have sensitized the Committee to potential problems in implementing protocols and to possible discrepancies between the consent forms and processes approved and those actually implemented. Although most of the discrepancies found during the consent form audit represented oversights and process errors rather than clearly unethical behaviour, they did suggest a lack of rigorous attention to detail.
Others have found that some research conducted on human subjects is not approved by a research ethics board and that some research protocols are changed after they have been approved; the research is thus not conducted according to the relevant ethical guidelines and the requirements of the research ethics board.6 In one study,2 investigators were found not to recognize the need for approval by the research ethics board of amendments to study protocols, and there were various discrepancies regarding the availability and completeness of consent forms and other records. Monitoring of informed consent has been the subject of several other reports.2,6,7,8,9 Direct monitoring of informed consent in one oncology study was found to be a feasible and important component of the research ethics committee's approach to studies that pose serious risk to subjects.9
Other aspects of protocols for which monitoring has been suggested include protocol compliance (e.g., compliance with eligibility criteria) and quality of data collected. These aspects would be more difficult to assess because monitors would have to be trained specifically for each protocol. We have found it reasonable instead to require that investigators specify in writing how they will avoid problems. For example, one investigator was asked to sign a checklist of eligibility criteria for each patient enrolled in the protocol, to help ensure that the investigator reviewed this information for each patient.
Variability in recall and understanding of research protocols on the part of research subjects has been previously documented.10,11 The reasons why patients consent to participate in studies and potential conflicts of interest (e.g., situations in which the patient's physician seeks consent for participation) require attention from research ethics boards.
Most of the investigators we surveyed supported monitoring, although the issue of who should pay was contentious. Possible solutions include asking investigators to make voluntary contributions (a strategy that was successful for one protocol at St. Mary's Hospital Centre, which was funded by a pharmaceutical company), asking investigators to include these costs in their research proposals and increasing the overhead rate charged to pharmaceutical companies. At our hospital it takes between 3 and 5 hours per protocol to monitor recruitment logs and to audit consent forms (including time needed for initial contact with the investigator to explain the monitoring and to follow up on the results). To this must be added the time needed to compile reports for the Research Ethics Committee. The resulting cost for each protocol is modest and could probably be accommodated within the budget of larger projects. The costs of monitoring unfunded projects would likely have to be borne by the institution.
Monitoring has become part of efforts by the St. Mary's Research Ethics Committee to continuously improve the quality of its reviews. On the basis of the results reported here, the committee has made or is planning several changes in its procedures. Approved consent forms are now stamped and dated to emphasize the importance of using the approved form. Investigators will be asked to provide information on any refusals to participate and on withdrawals from protocols. The hospital's patient representative, who is named on consent forms as the person to be contacted by potential subjects to discuss their rights as research subjects, will provide information to the Research Ethics Committee on the nature of these contacts. Educational activities, including workshops on ethical aspects of research and the procedures of the Research Ethics Committee, have been offered for hospital investigators and research assistants.
In conclusion, the types of monitoring we conducted are feasible and reveal many problems in obtaining informed consent and other aspects of protocols that would be of potential interest to research ethics boards. Our experience may not be generalizable to other hospitals, particularly those with research institutes or a large number of research protocols. Nevertheless, others may wish to adapt certain aspects of our program to monitor the research in their institutions.
Footnotes
-
This article has been peer reviewed.
Contributors: Dr. McCusker analysed the survey responses and was the lead writer for the paper. Dr. Kruszewski assisted in writing the paper and performed the literature review. Ms. Lacey and Mr. Schiff, as cochairs of the Research Ethics Committee, developed and implemented the monitoring program and contributed to writing the paper.
Acknowledgments: We are grateful to April Collins, Research Administrative Coordinator at St. Mary's Hospital Centre, for providing statistics from the review of protocols, and to Lorraine Merriot, Johanne Laplante and Dominique Proulx for their assistance with monitoring activities and data collection for this paper.
Competing interests: None declared.
Reprint requests to: Dr. Jane McCusker, Clinical Epidemiology and Community Studies, St. Mary's Hospital Centre, 2508-3830 Lacombe Ave., Montreal QC H3T 1M5; fax 514 734-2652; [email protected]
References
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.