Assessing the quality of clinical practice guidelines

Assessment of the quality of clinical practice guidelines (CPGs) is essential, and a systematic review of these guidelines is strongly encouraged. To that end, Ian Graham and colleagues have taken an important step in trying to assess the quality of a number of Canadian CPGs.1

The authors state that CPGs are “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” However, 38 (17.5%) of the 217 “guidelines” reviewed for this article were letters produced by the Therapeutics Initiative. In our opinion, these letters should have been considered as systematically developed reviews of evidence, not CPGs.

Since 1994, 41 Therapeutics Initiative letters have been produced. In general, our objective is to provide clinicians with an assessment and synthesis of published (and, whenever possible, peer-reviewed) evidence from clinical trials. The information is primarily, although not exclusively, a synopsis of the evidence from randomized controlled trials. We occasionally make recommendations in our letters for the “optimal” clinical use of drugs when supported by evidence from randomized controlled trials. We hope that clinicians combine these synopses with clinical judgement when making decisions about drug therapy for individual patients. Evidence from randomized controlled trials should also be the underpinning of all CPGs; however, in order to generate bottom-line recommendations, authors of CPGs often have to resort to less rigorous evidence or clinical opinion or both, owing to the limited availability of high-quality evidence.

For example, the most recent Canadian guidelines for initial management of community-acquired pneumonia furnish an exhaustive review of this condition; however, there is no clear evidence from randomized controlled trials favouring any one antibiotic regime.2 In fact, the authors urge recognition that “these recommendations [for the selection of antibiotics] are derived by the consensus of experts and not entirely based on evidence from randomized clinical trials.”2

These important differences between systematically developed recommendations (e.g., CPGs) and systematically developed reviews of the evidence (e.g., Therapeutics Initiative letters, Cochrane Library) render an appraisal instrument for clinical guidelines less appropriate for assessing systematically developed reviews of evidence. Many of the criteria in Table 1 of the article by Graham and colleagues are not relevant to the latter process. Nonetheless, these authors have presented a number of criteria relevant to both guidelines and systematic reviews of the evidence; we will review these to see if there are areas in which we can improve.