AGREE 1,2 focuses on methodologic issues about guideline development and reporting. Although important, these issues are insufficient to ensure that recommendations are appropriate and valid, because methodologic rigour and validity are not necessarily correlated. 3
In an assessment of pharyngitis guidelines by a panel familiar with AGREE, 4 guidelines not citing European trials or the Cochrane review had higher AGREE scores than most other guidelines. Only 3 of 23 items in the AGREE instrument assesses the description of the evidence, although each item has the same weight. We therefore decided not to use AGREE in our comparison; we evaluated the evidence with an analysis of the references of the 10 existing guidelines. We concluded that high AGREE scores do not guarantee that the selection of the evidence has been adequately performed — even with the new item #9 about strengths and limitations of the evidence — and that different items may have different relevance to the validity of the recommendations.
In a study on mass colorectal screening, guidelines were found to be equally valid regarding their (different) recommendations related to fecal occult blood tests, but no relation could be found between their methodologic AGREE quality and their content validity. 5 Although AGREE is important in developing guidelines, the World Health Organization has also recognized the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. 6
Footnotes
-
For the full letter, go to: www.cmaj.ca/cgi/eletters/182/11/1151#595071