Guideline groups should make recommendations even if the evidence is considered insufficient
============================================================================================

* Ignacio Neumann
* Holger J. Schünemann

KEY POINTS
*   Health guidelines are useful in the context of sparse or doubtful evidence, sometimes called insufficient evidence, but not all guideline panels issue recommendations in this context.

*   Without explicit recommendations, guideline panels miss an opportunity to support clinicians and patients, who do not have the time or the resources available to guideline panellists.

*   Guideline panels should use explicit frameworks to support the appropriate reporting of all considerations that determined the decisions and make explicit all challenges faced in making specific recommendations, but they must make recommendations.

*   Making conditional recommendations based on sparse evidence does not preclude, and explicitly encourages, further relevant research.

Clinicians and patients often face situations in which the evidence to support a proposed intervention is sparse or doubtful — sometimes called “insufficient evidence.” Some guideline developers are reluctant to make concrete recommendations when evidence is deemed insufficient. For example, the US Preventive Services Task Force has a special category called “I statement” (for insufficient evidence),1 which allows them to avoid recommending an intervention that may be proven ineffective or harmful in the future. Guideline panellists may perceive recommendations based on little or no evidence as “not evidence based” and therefore unfounded. Some have argued that not making recommendations in circumstances when the evidence is low quality highlights knowledge gaps and may therefore stimulate future research.2 However, we argue that developers of health guidelines should make recommendations regarding the best course of action when evidence is sparse or doubtful, as they are best equipped to do so, and evidence shows that health professionals want them to.3

Although the cautious approach taken by guideline panels is understandable, a recent randomized trial, in which more than 500 clinicians were presented with clinical scenarios informed by sparse evidence, found that more than 80% of clinicians preferred receiving a recommendation.2 Clinicians strongly prefer having recommendations in the context of uncertainty.

Guideline panellists have time and resources that users of recommendations do not have. In an optimal guideline panel, methodologists conduct systematic reviews and present the available evidence to a panel composed of leading clinical or public health experts and other stakeholders such as patients. After careful review of the evidence regarding benefits and harms of interventions, as well as patients’ values, acceptability, feasibility, impact on health equity and resource considerations, the panel recommends the course of action in which it believes the desirable consequences are most likely to outweigh the undesirable ones.

Clinicians need to make decisions, and without explicit recommendations, they and their patients can be left adrift. For example, in a recent guideline, the International Liaison Committee on Resuscitation considered there to be too much uncertainty to make a recommendation about the use of prophylactic antiarrhythmic drugs after a successfully treated cardiac arrest.4 What should clinicians do in this situation? They might infer that because the guideline panel avoided issuing a recommendation, antiarrhythmic drugs should not be used. However, observational data suggest otherwise. In a cohort of 1721 patients, the administration of prophylactic lidocaine upon return to spontaneous circulation was associated with a reduction of recurrent cardiac arrest and improvement of survival.5 If the guideline panel had considered these data and found them relevant to the decision about the use of antiarrhythmic drugs, a conditional recommendation in favour of lidocaine, acknowledging the low certainty of the evidence, would have been helpful to users of the guideline.

Conversely, just presenting options without guidance may lead to excessive use of a treatment. Consider the use of stem cells in Parkinson disease, a novel treatment being offered in many places in the world. The evidence about the effects of this treatment is very limited; only 2 small trials with 74 patients in total have been published.6,7 Further, in 1 of the trials, 2 patients allocated to stem cell treatment experienced a major intracerebral complication. Simply outlining the available options in this scenario could lead to early adoption of an intervention that may expose patients to harm. A better option would have been a conditional recommendation against stem cell treatment until further research becomes available.

Providing a recommendation need not discourage future research. For example, in 2013, the World Health Organization (WHO) developed a guideline about a new medication to support treatment of multidrug-resistant tuberculosis: bedaquiline.8 The panel made a conditional recommendation in favour of adding the drug to a WHO-recommended regimen based on very low-certainty evidence, using the available trial data from 132 patients that suggested improvement in or cure from multidrug-resistant tuberculosis. The guideline report explicitly described the challenges in making the recommendation. The anticipated benefit of a 26% absolute increase in the likelihood of cure found in 1 trial had to be balanced against a statistically significant absolute increase in death of 10% (based on a total of 10 deaths in 160 patients available for the mortality outcome).9 The guideline also explicitly called for further research, leading to individual patient data meta-analyses of postmarketing and field studies from thousands of patients, which were used to update these guidelines 3 years later, coordinated by the WHO.9 Making the recommendation in the 2013 guideline did not prevent important and necessary research from taking place; on the contrary, it fostered it.

Guideline panels should certainly make every effort to reduce uncertainty by commissioning or conducting systematic reviews and involving experienced methodologists in their process. They should also consider factors other than the benefits and harms of the interventions, such as patient perspectives and cost, when making their recommendations. Guidelines cannot be considered trustworthy without such precautions,10 and guideline panels must be careful and transparent when making recommendations in the context of uncertainty.11 Explicit frameworks12 may help to support the appropriate reporting of all the considerations that determined the decisions and make explicit all challenges faced by the panel. Transparent frameworks may also help clinicians and patients better understand what is known and not known about the available evidence, and help them consider factors such as the burden of treatment and patients’ values.

Because clear practice recommendations are highly valued by clinicians and may prevent the use of unproven interventions, we believe that guideline panellists should always endeavour to make recommendations in the context of uncertainty. Doing so should not preclude, and may encourage, future relevant research.

## Footnotes

*   **Competing interests:** Holger Schünemann reports that he is the co-chair of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group. No other competing interests were declared.

*   This article has been peer reviewed.

*   **Contributors:** Both authors contributed to the conception and design of the work, drafted the manuscript, revised it critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

## References

1.  Grade definitions. Rockville (MD): U.S. Preventive Services Task Force; 2018. Available: [www.uspreventiveservicestaskforce.org/Page/Name/grade-definitions](http://www.uspreventiveservicestaskforce.org/Page/Name/grade-definitions) (accessed 2019 Jan. 28).
    
    

2.  Bretthauer M, Kalager M. When no guideline recommendation is the best recommendation. Lancet 2018;392:898–9.
    
    

3.  Neumann I, Alonso-Coello P, Vandvik PO, et al. Do clinicians want recommendations? A multi-center study comparing evidence summaries with and without GRADE recommendations. J Clin Epidemiol 2018;99:33–40
    
    

4.  Soar J, Donnino MW, Maconochie I, et al.ILCOR Collaborators. 2018 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science with treatment recommendations summary. Circulation 2018;138:e714–30.
    
    

5.  Kudenchuk PJ, Newell C, White L, et al. Prophylactic lidocaine for post resuscitation care of patients with out-of-hospital ventricular fibrillation cardiac arrest. Resuscitation 2013;84:1512–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.resuscitation.2013.05.022&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23743237&link_type=MED&atom=%2Fcmaj%2F192%2F2%2FE23.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000328654800015&link_type=ISI) 

6.  Freed CR, Greene PE, Breeze RE, et al. Transplantation of embryonic dopamine neurons for severe Parkinson’s disease. N Engl J Med 2001;344:710–9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM200103083441002&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11236774&link_type=MED&atom=%2Fcmaj%2F192%2F2%2FE23.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000167305300002&link_type=ISI) 

7.  Olanow CW, Goetz CG, Kordower JH, et al. A double-blind controlled trial of bilateral fetal nigral transplantation in Parkinson’s disease. Ann Neurol 2003; 54:403–14.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/ana.10720&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=12953276&link_type=MED&atom=%2Fcmaj%2F192%2F2%2FE23.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000185158100018&link_type=ISI) 

8.  The use of bedaquiline in the treatment of multidrug-resistant tuberculosis. Interim policy guidance. Geneva: World Health Organization; 2013. Available: [http://apps.who.int/iris/bitstream/10665/75146/1/9789241548441\_eng.pdf](http://apps.who.int/iris/bitstream/10665/75146/1/9789241548441_eng.pdf) (accessed 2014 Dec. 8).
    
    

9.  Report of the Guideline Development Group Meeting on the use of bedaquiline in the treatment of multidrug-resistant tuberculosis: a review of available evidence (2016). Geneva: World Health Organization; 2017.
    
    

10. 1.  Graham R, 
    2.  Mancher M, 
    3.  Miller Wolman D, 
    4.  et al.
    
    Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical practice guidelines we can trust. Washington (D.C.): National Academies Press; 2011.
    
    

11. Schünemann HJ, Zhang Y, Oxman AD. Expert Evidence in Guidelines Group. Distinguishing opinion from evidence in guidelines. BMJ 2019;366:l4606
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNjYvanVsMTlfNC9sNDYwNiI7czo0OiJhdG9tIjtzOjIwOiIvY21hai8xOTIvMi9FMjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

12. Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ 2016;353:i2016.
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTMvanVuMjhfOS9pMjAxNiI7czo0OiJhdG9tIjtzOjIwOiIvY21hai8xOTIvMi9FMjMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)