Variations in mortality rates among Canadian NICUs —and anonymous reporting

The position taken in the recent commentary “What's in a name? Reporting data from public institutions”1 is thoughtful, provocative and, in an ideal world, one with which we would have no argument. In an ideal world, all research tools would already be fully validated, research-funded institutions would in any case be required to report routinely a wide variety of indicators on their performance, and the bodies to whom they were accountable (be it regions or provinces) would in turn be required to report to the public on the relative performance of their publicly funded institutions. Under such circumstances, there would be no question that a researcher should identify all organizations involved in his or her research. But we're not there yet, and we need to deal with reality.

At least 3 practical issues deserve to be highlighted. First is the validity of measurement tools.

As the commentary acknowledges, research involves the continuous development and refinement of new measurement and evaluation tools; each new instrument requires time and experience before it can be validated. Institutions and practitioners involved in research using these nonvalidated instruments during the development and testing phases, may, understandably, not be comfortable with committing themselves a priori to being identified. The risk of misleading results should not be underestimated. One could cite a number of high-profile examples of the implications for institutions of “too-early” exposure. Thus, this seems a reasonable and defensible position for these institutions. Whether institutions would be prepared to participate in research without remaining anonymous if some acceptable and explicit validity criteria could be established is an interesting question (but one for which neither we, nor CMAJ, currently has answers).

The second issue relates to the grandfathering of existing agreements. Research that is already completed, or currently in progress, may have been initiated without expectations or understandings of the sort articulated in the CMAJ editorial. Hence, even if the issues raised in the first paragraph were all resolved, it seems only fair that these studies be “grandfathered” if that is the wish of the participating institutions. Until such time as the agencies that fund research require disclosure at the time of project approval, it is inappropriate for CMAJ to impose this expectation after the fact.

The third concern is the need to strengthen the requirements placed on publicly funded institutions. Our view is that this issue cannot be addressed narrowly, in isolation from the more general issues of accountability and public expectations imposed on institutions and their providers. It makes no sense for a journal to impose disclosure conditions at the end of a process that are inconsistent with terms and conditions “upstream.” If the provincial governments and regional health authorities that fund the institutions in question do not require them to participate in research with a disclosure agreement, then why should they voluntarily expose themselves to the danger of “looking bad” that is inherent in comparative institutional studies. CMAJ merely wishing them to be “good citizens” will not make it so.

These issues identify 3 specific classes or categories of “actors” — care providers and institutions; research funding agencies; and care governance and funding bodies — that would be required to revise their current operations for a CMAJ-type editorial policy to make sense.

We recognize that we are offering these observations unencumbered by any evidence on institutions' actual views on these matters. We need data from a survey of institutional and care-provider attitudes toward revealing their identities in the context of prospective research studies that address clinical or organizational management practices. To what extent would widespread implementation of the CMAJ editorial position simply choke off researchers' access to the only settings in which publicly accountable performance measures can be validated and the data collected? Perhaps this is all a “storm in a teacup,” and most institutions and providers would already be comfortable participating even with the understanding that they would be identified. But we suspect not.

We offer these thoughts as part of the debate on the important issues that your commentary raises and in the hope that some of the practical issues will be addressed soon. In the interim, neither CHSRF nor CIHR is planning to introduce a new “non-anonymity” condition on funding, at least in situations where a major part of a research project is development and validation of new measures, or the application of existing instruments in new contexts or for new purposes.

Jonathan Lomas Executive Director Canadian Health Services Research Foundation (CHSRF) Ottawa, Ont. Morris Barer Scientific Director Institute of Health Services and Policy Research Vancouver, BC