Registration of observational studies: Is it time?

Why register observational studies?

Much of the rationale for the prospective registration of clinical trials 9 applies to the registration of observational studies (Table 1). 7 For example, observational studies in which researchers acquire data directly from human participants entail ethical obligations to participants, even though such research generally involves less risk than interventional studies. 10 These obligations include oversight by ethical review boards, informed consent, and public release of the study findings to advance biomedical knowledge. As with clinical trials, incomplete reporting of observational studies has been documented. 3 Some researchers suggest that observational studies are also at increased risk for publication bias or other types of bias, including misrepresentation of prespecified analyses or phenotype definitions. 2^,4 Such biases are a concern because they undermine the validity of observational studies, which are an important component of the medical evidence base in areas of public health, such as detection of rare adverse events. 1^,16

Observational studies of medications and devices are playing a more visible role at the United States Food and Drug Administration (FDA) (Table 2). For instance, the FDA posted an “Early communication about an ongoing safety review” 18 in response to a published observational study associating abacavir and didanosine with an increased risk of cardiovascular and cerebrovascular events. 19 The authors of the study noted that, although a randomized controlled trial is necessary to show a causal association, such a trial design is “unlikely to be feasible,” given that it would require more than 10 000 participants to be followed for at least two years. In addition, there may be ethical concerns in conducting a randomized controlled trial if harms are expected. Given that this and other associations between marketed products and possible harms are likely to be investigated further using observational studies, a registry containing summary protocol information would allow researchers to track such studies from initiation to completion. Such a tool could be useful to researchers who are evaluating the current evidence, considering initiating similar studies, identifying gaps in research or seeking collaborators. 15 Similarly, a database of summary results could improve access to information about published and unpublished observational study analyses (whether prespecified or post hoc), thereby mitigating publication bias and incomplete reporting of results.

Given these potential benefits, observational studies are already being registered for various reasons, and there is increasing attention given to this practice. For example, the state of Maine requires registration and reporting of results of postmarketing observational studies of medications and biological products marketed in Maine 20 and the corporate policies of some drug manufacturers address disclosure of observational studies. 21 Recently, an international workshop was held on the topic, 6 and several medical journals published editorials on the registration of observational studies. 7^,8^,22 Some have suggested that ethics review boards should require prospective registration of any study involving human participants, 12 whereas others have proposed that observational studies under the mandate of the FDA should be subject to the same requirements for registration and reporting of results as those for clinical trials. 17^,23 The European Medicines Agency recently issued a work plan to create a registry of post-authorization safety studies that would include observational studies, 24 and a draft report by the Agency for Healthcare Research and Quality considered the utility of creating a “registry of patient registries.” 25 With growing interest in this topic, an understanding of current practices in registration is useful.

Where can observational studies be registered?

An increasing number of clinical trial registries exist and, despite their names, some include observational studies. For example, ClinicalTrials.gov (http://clinicaltrials.gov), established by the US National Library of Medicine on behalf of the US National Institutes of Health, is the largest publicly accessible registry, with over 87 000 ongoing and completed interventional and observational studies as of March 2010. There are 10 registries in the World Health Organization’s Registry Network, which range in size from 25 to more than 8400 registered studies as of March 2010. Of these registries, five explicitly state that observational studies can be registered (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.092252/DC1). A structured search tool for identifying observational studies is provided at ClinicalTrials.gov, where such studies represent 17% (n = 14 595) of all registrations, and at the Australian New Zealand Clinical Trials Registry, where 5% (n = 214) of registered studies are observational.

This cursory review of international registries suggests that the basic technical infrastructure for the registration of observational studies already exists. Closer examination is required to determine whether such registries are meeting the needs of users — from the perspective of both sponsors entering data and researchers (among others) using the display and retrieving studies. The remainder of this commentary will use the experience of ClinicalTrials.gov to characterize some current practices in the registration of observational studies and identify potential issues.

Registration of observational studies at ClinicalTrials.gov

ClinicalTrials.gov has accommodated the registration of observational studies since its launch in February 2000. It standardizes key attributes of study protocols using an established set of mandatory and voluntary data elements (http://prsinfo.clinicaltrials.gov/definitions.html). Sponsors must designate a study as observational or interventional using a mandatory data element that indicates type of study before they complete other design-specific data elements. Observational studies are found on the public site by selecting “Observational Studies” from the “Study Type” drop-down menu on the advanced search form at ClinicalTrials.gov (http://clinicaltrials.gov/ct2/search/advanced).

In October 2007, modifications were made to the design-specific data elements used for registering observational studies. These changes were strongly influenced by protocol-related items in the statement Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). 14 The STROBE initiative aims to improve the reporting of observational studies by specifying a core set of items to include in a publication that would allow a reader to follow “what was planned, done, and found.” 14 Consistent with the aims of the STROBE initiative, registration facilitates prospective documentation of key prespecified study attributes. In September 2008, Clinical-Trials.gov added a database of summary results that allows for reporting of many of the results-related items identified in the STROBE initiative, such as information on study participants (e.g., numbers of participants starting and completing the study, baseline demographic characteristics), outcome measures (including appropriate statistical analyses) and adverse events, if relevant.

After the design-specific data elements for observational studies were updated, prospective cohort studies (n = 2617) comprised the largest segment of observational studies registered in ClinicalTrials.gov between November 2007 and March 2010 (n = 7735) (Appendix 2, available at www.cmaj.ca/cgi/content/full/cmaj.092252/DC1). Trends in observational studies registered in ClinicalTrials.gov from February 2000 to March 2010 include the following:

Gaps in knowledge

Based on our experience, we believe that the existing model for and system of registration of clinical trials can be applied to observational studies. Although ClinicalTrials.gov is able to accommodate a range of observational study designs, those that involve collection of primary, prospective data may fit this model more easily because such studies are most similar to interventional studies. In each type of study, data are generally collected prospectively from well-defined groups of individuals (e.g., exposed and nonexposed) according to a protocol. As previously noted, prospective cohort studies were the most frequently registered in ClinicalTrials.gov (Appendix 2). Whether and how other types of observational studies (e.g., retrospective data collections, secondary data analyses and evaluations of data from patient registries) should be represented in a study registry requires further consideration. For example, patient registries are established to collect a defined set of standardized variables about specific patient populations, but they may or may not have specific protocols with prespecified plans for analysis. Although hundreds of patient registries have been successfully registered (e.g., http://clinicaltrials.gov/search/term=registry), when they should be registered and which attributes should be represented could be further elucidated. The Agency for Healthcare Research and Quality is currently considering these issues. 25

Other challenges related to the registration of observational studies include difficulties in delineating individual studies from a broad plan for data analysis, and whether and how sub-studies and secondary studies or analyses using the same prospective data set should be registered. In addition, data elements and definitions related to data collection for study variables, 14 as well as the specific structure for reporting results of observational studies, may require further consideration. For example, although the ClinicalTrials.gov registry currently collects information on outcome measures, it may also be useful to prespecify other variables (e.g., baseline characteristics) that will be used to characterize a cohort or that will be included in an analysis. 26

Some have questioned the practicality of registering observational studies because of the difficulty of documenting all hypotheses and have expressed concern that prespecification of all hypotheses will inhibit exploratory research. 13^,26 Other approaches to publicly documenting original study design and variables already exist, such as the publication of cohort profiles in the International Journal of Epidemiology. 27 Registration of observational studies may be complementary to such publications, given that the registry record is a standard format, is broadly accessible and can be updated over time and given that links or cross-references between the registry and publication could be established.

Next steps: Is it time to register observational studies?

There is interest in the prospect of registering observational studies, as reviewed earlier and as shown by the number of such studies registered at ClinicalTrials.gov. We believe that making both observational and interventional studies available in a single registry will provide researchers and others with a more comprehensive view of the growing evidence base. Based on the clinical trial experience, there is a pre-existing (but evolving) model and infrastructure for registration, and study sponsors are already registering observational studies on a voluntary basis. Given that much of the ethical and scientific rationale for registration of interventional studies appears to apply to prospective observational studies, we believe that further discussion among stakeholders (e.g., researchers, regulatory authorities and the public) is warranted. Such discussion is necessary to address specific issues regarding implementation, including assessing and defining common data elements that would meet the needs of the stakeholders. In the future, the data elements for registration of observational studies in ClinicalTrials.gov could be modified to better meet these needs.