Objectives: It is not clear if good methodological quality in developing practice guidelines (GLs) necessarily leads to valid recommendations that, when implemented, are more likely to improve the balance between benefits and harms/costs. We assessed whether or not there is a link between methodological quality and recommendation validity in GLs for the use of fecal occult blood test (FOBT) as a screening test for colorectal cancer (CRC) in the average-risk population.
Methods: We systematically searched for such practice GLs published in English or in French within the last 7 years. Our inclusion criteria limited the initial 36 GLs to 12. Scores for methodological quality based on the AGREE criteria were attributed to each of these 12 GLs. Likewise, we searched for meta-analysis and other systematic reviews (SRs) addressing the issue, and we selected for inclusion 8 SRs that met basic quality criteria according to the Critical Appraisal Skills Programme (CASP) of the National Health Service of the United Kingdom (NHS). We used the results and conclusions of these 8 SRs to establish the validity of recommendations made in the 12 included GLs.
Results: Regarding methodological quality, the GLs were labeled either "strongly recommend" (n=3), "recommend with provisos or alterations" (n=3), "would not recommend" (n=2), or "unsure" (n=4). The nine GLs recommending for, as well as the three GLs recommending against, mass-screening are equally valid, because the former base their recommendation on the fact that this can decrease CRC-mortality, whereas the latter base their recommendation on the facts that: (1) this procedure would be too expensive and/or not adapted to their local organization of care, and (2) to a lesser extent, the balance between benefits and harms is not crystal-clear from an individual patient perspective.
Conclusion: The fact that the 12 GLs fell short of basic quality criteria confirms many previous observations in various areas of medicine. Because the 12 GLs were found to be equally valid regarding their FOBT-related recommendations, no relation could be found between their methodological quality and their content validity.