Direct and indirect randomized trials of screening: the A's and D's of evidence-based clinical practice guidelines
==================================================================================================================

* Jeffrey Mahon

The recommendation by the Canadian Diabetes Association to screen all asymptomatic Canadians over 45 years of age for diabetes mellitus has generated controversy.[1–3] This recommendation arose from an evidence-based approach to formulate clinical practice guidelines and was accorded the qualifier "grade D" on the basis of "expert consensus." Kenneth Marshall2 argued that there is no evidence that such screening will decrease morbidity and mortality; Hertzel Gerstein and Sara Meltzer3 disagreed. One issue underlying the controversy is that Marshall implicitly requires that a direct randomized trial be done, while Gerstein and Meltzer are willing to accept evidence from indirect clinical trials. How do these trials differ? Why is the distinction important? And should (and can) a direct randomized trial be conducted before accepting this recommendation?

Screening tests are not only diagnostic; they can also be viewed as interventions and must therefore satisfy 3 evaluative hurdles before they are accepted. They must be efficacious (i.e., they must result in clinically important health gains for those being tested), their benefit must exceed the negative effects of testing, and the net benefit (total benefit less total harm) must justify the economic costs of testing and treatment. Efficacy, the foremost hurdle, can be assessed in observational studies or in randomized trials. The latter are preferred because they provide better control on inherent biases in observational studies of screening tests that lead to inflated estimates of efficacy.

Randomized trials of screening can be either direct or indirect in design (Fig. 1). In a direct trial individuals are randomized to be screened or not; those who screen positive are treated, those who screen negative are not treated, and all 3 groups are followed for the outcome of interest. For example, several direct randomized trials have found that regular fecal occult-blood tests reduced colon cancer mortality,4 and direct randomized trials of mammography screening for women 50-69 years of age reported a reduction in breast cancer mortality.5

![Figure1](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/162/7/1002/F1.medium.gif)

[Figure1](http://www.cmaj.ca/content/162/7/1002/F1)

Fig. 1: Methodology for conducting direct and indirect randomized trials.

In contrast, in an indirect trial a population is screened, and those who test positive are randomized to be treated or not. The efficacy of measuring low-density lipoprotein cholesterol levels on the primary prevention of atherosclerotic complications in high-risk individuals,6 of measuring bone mineral density by dual-energy x-ray absorptiometry on preventing fractures in post-menopausal women,7 and of measuring blood pressure in asymptomatic adults on preventing strokes8 was established in indirect trials. For example, among individuals found to have elevated low-density lipoprotein cholesterol levels through screening, those who were then randomized to a pravastatin group were significantly less likely to die of cardiovascular disease than those randomized to placebo.6

In schemes to formulate evidence-based clinical practice guidelines, the strength of the recommendations supporting screening tests varies according to the direct or indirect nature of the evidence. Thus, strong qualifiers such as "level 1 evidence" and "grade A recommendation" tend to be reserved for direct trial evidence,9 whereas results of indirect trials usually engender weaker terms such as "expert consensus" and "grade D recommendation."1 Stronger recommendations can be made on the basis of direct trial evidence because exposure to the screening test itself is the only difference between the 2 randomized groups. Thus, we can be more confident that differences in outcomes - both good and bad - are explained by the act of screening. This is especially valuable if the screening test has the potential to cause the same outcome it prevents (e.g., concern, since refuted, that irradiation during mammography causes breast cancer). In addition, because direct trials include an untested randomized control group, the risk of serious negative effects associated with the screening test and its sequelae (e.g., colon perforation in patients who undergo colonoscopy after a positive fecal occult-blood test) can be more readily assessed. A more conservative endorsement of a screening test in the absence of direct trial evidence is therefore often justified.

However, there are instances where indirect trials can demonstrate that a screening test is efficacious, and in some cases, that a subsequent direct trial is unnecessary. If the indirect trial uses the same screening test in similar populations and at similar points in the disease as would be used in the direct trial, if it is unlikely that the test and its sequelae will cause the outcome it is expected to prevent, and if there is consensus that the clinical importance of the outcomes that were prevented exceed the screening test's harm, then a direct trial becomes a difficult, if not unethical, proposition. This is because the direct trial randomizes people who are at risk to a "no screening" group; these individuals are therefore deprived of the therapy proven in the indirect trial to prevent a clinically important outcome. It is for this reason that it is unlikely direct trials of serum lipid or blood pressure measurements in asymptomatic middle-aged and older persons or of dual-energy x-ray absorptiometry scanning in asymptomatic post-menopausal women will ever be done.

This is precisely the situation when considering diabetes testing in people 45 years of age and older, irrespective of other risk factors for diabetes or its complications. Indirect randomized trials in those with type 2 diabetes have identified treatments that prevent the progression of retinopathy, neuropathy and incipient nephropathy.[10–13] Another indirect randomized trial14 found that lowering diastolic blood pressure in people with diabetes to levels that would otherwise not define hypertension (i.e., < 90 mm Hg) reduced major cardiovascular events. Moreover, the findings of at least 1 of these indirect trials10 are likely to be applicable to those with type 2 diabetes who will be identified and treated earlier than usual if the Canadian Diabetes Association's testing recommendations are followed. When it is noted that 5% of people between 45 and 54 years of age have unrecognized diabetes15 and up to one-third of those people already have microvascular complications that would respond to improved glycemic control,11 testing for type 2 diabetes after the age of 45 becomes reasonable. This does not mean that the negative consequences and economic costs associated with doing so are trivial; Marshall's concerns about these issues are significant.2 However, the clinical impact and economic savings from preventing more advanced microvascular complications16 have made many doubt that the negative effects of testing exceed these benefits.

In the future evidence-based recommendations for clinical practice guidelines on the use of screening tests may incorporate qualifiers to indicate whether the recommendation is supported by indirect or direct randomized trials. In the meantime, the recommendation of the Canadian Diabetes Association to test asymptomatic people for diabetes beginning at 45 years of age, although one that remains qualified by "grade D, expert consensus" because it is based on indirect trials, is justified. I doubt that direct randomized trials of screening for diabetes will be conducted in this and other high-risk groups. We will therefore have to rely on data from indirect trials and make the best clinical judgements we can. To the extent that practitioners and patients need not concur with experts, each can, and should, decide for themselves about whether to be tested. Whatever the decision, it is important that both the proven benefits of such testing, identified in indirect randomized trials, as well as the potential negative effects be weighed.

Competing interests: Dr. Mahon was a member of the Canadian Diabetes Association Expert Subcommittee that developed recommendations for the management of diabetes in Canada.

## Acknowledgments

**For the leadership and management skills necessary to function effectively** 

Approved for RCPSC, CFPC and AAFP study credits

**PMI-1/PMI-2** 

June 4-6/June 7-9, 2000

Sept. 24-26/Sept. 27-29, 2000

**PMI-3/PMI-4** 

Apr. 30-May 2/May 3-5, 2000

Nov. 5-7/Nov. 8-10, 2000

**PMI Refresher** 

Oct. 20-22, 2000

Château Laurier, Ottawa; The Queen Elizabeth, Montréal

Pillar & Post Inn, Niagara-on-the-Lake, Ont. Sutton Place Hotel, Vancouver

Sutton Place Hotel, Vancouver

**In-house PMI** 

A practical, cost-effective and focused training opportunity held on site for leaders and managers

For information: tel 800 663-7336 or 613 731-8610; x2319 (PMI) or x2261 (In-house PMI) michah{at}cma.ca; [www.cma.ca/prodev/pmi](http://www.cma.ca/prodev/pmi)

**Canadian College of Health Service Executives Collège canadien des directeurs de services de santé** 

## Footnotes

*   Correspondence to: Dr. Jeffrey Mahon, London Health Sciences Centre, 339 Windermere Rd., London ON N6A 5A5; fax 519 663-3211; [Jl.Mahon@lhsc.on.ca](http://Jl.Mahon@lhsc.on.ca)

## References

1.  1. Meltzer S, Leiter L, Daneman D, Gerstein HC, Lau D, Ludwig S, et al. 1998 clinical practice guidelines for the management of diabetes in Canada. CMAJ 1998;159(8 Suppl):1S-29S.
    
    

2.  2. Marshall KG. The folly of population screening for type 2 diabetes. CMAJ 1999;160(11):1592-3.
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJjbWFqIjtzOjU6InJlc2lkIjtzOjExOiIxNjAvMTEvMTU5MiI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xNjIvNy8xMDAyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

3.  3. Gerstein HC, Meltzer S. Preventive medicine in people at high risk for chronic disease: the value of identifying and treating diabetes. CMAJ 1999;160(11):1593-5.
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJjbWFqIjtzOjU6InJlc2lkIjtzOjExOiIxNjAvMTEvMTU5MyI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xNjIvNy8xMDAyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  4. Fletcher RH. Should all people over the age of 50 have regular fecal occult-blood tests? If it works, why not do it? N Engl J Med 1998;338:1153-4.
    
    

5.  5. Miller AB. An epidemiological perspective on cancer screening. Clin Biochem 1995;28:41-8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/0009-9120(94)00069-8&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7720226&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1995QG33700004&link_type=ISI) 

6.  6. Shepherd J, Cobbe SM, Ford I, Isles CG, Lorimer AR, MacFarlane PW, et al. Prevention of coronary heart disease with pravastatin in men with hypercholesterolemia. West of Scotland Coronary Prevention Study Group. N Engl J Med 1995;333:1301-7.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199511163332001&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7566020&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1995TE36500001&link_type=ISI) 

7.  7. Karpf DB, Shapiro DR, Seeman E, Ensrud KE, Johnston CC, Harris ST, et al. Prevention of nonvertebral fractures by alendronate. A meta-analysis. JAMA 1997;277:1159-64.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1997.03540380073035&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9087473&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997WR34200037&link_type=ISI) 

8.  8. Ogilvie RI, Burgess ED, Cusson JR, Feldman RD, Leiter LA, Myers MG. Report of the Canadian Hypertension Society Consensus Conference: 3. Pharmacologic treatment of essential hypertension. CMAJ 1993;149:575-84.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8364814&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 

9.  9. Woolf SH, Battista RN, Anderson GM, Logan AG, Wang E, and the Canadian Task Force on the Periodic Health Examination. Assessing the clinical effectiveness of preventive maneuvres: analytic principles and systematic methods in reviewing evidence and developing clinical practice recommendations. J Clin Epidemiol 1990;43:891-905.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/0895-4356(90)90073-X&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=2145399&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1990EC89200007&link_type=ISI) 

10. 10. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet 1998;352:837-53.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(98)07019-6&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9742976&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000075857300006&link_type=ISI) 

11. 11. Ferris F. Early photocoagulation in patients with either type 1 or type 2 diabetes. Trans Am Ophthalmol Soc 1996;94:505-37.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8981711&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 

12. 12. Ohkubo Y, Kishikawa H, Araki E, Miyata T, Isami S, Motoyoshi S, et al. Intensive insulin therapy prevents the progression of diabetic microvascular complications in Japaness patients with non-insulin-dependent diabetes mellitus: a randomized prospective 6-year study. Diabetes Res Clin Pract 1995;28:103-17.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/0168-8227(95)01064-K&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7587918&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1995RR63400004&link_type=ISI) 

13. 13. Ravid M, Lang R, Rachmani R, Lishner M. Long-term renoprotective effect of angiotensin-converting enzyme inhibition in non-insulin-dependent diabetes mellitus: a 7-year follow-up study. Arch Intern Med 1996;156:286-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/archinte.1996.00440030080010&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8572838&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1996TU54500007&link_type=ISI) 

14. 14. Hansson L, Zanchetti A, Carruthers GC, Dahlof B, Elmfeldt D, Julius S, et al. Effects of intensive blood-pressure lowering and low-dose aspirin in patients with hypertension: principal results of the Hypertension Optimal Treatment (HOT) randomised trial. Lancet 1998;351:1755-62.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(98)04311-6&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9635947&link_type=MED&atom=%2Fcmaj%2F162%2F7%2F1002.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000074197600008&link_type=ISI) 

15. 15. Roman SH, Harris MI. Management of diabetes mellitus from a public health perspective. Endocrinol Metab Clin North Am 1997;26(3):443-74.
    
    

16. 16. Eastman RC, Dong F, Javitt JC, Manninen D, Herman WH, Zbrozek AS, et al. Model of complications of NIDDM. II. Analysis of the health benefits and cost-effectiveness of treating NIDDM with the goal of normoglycemia. Diabetes Care 1997;20:735-44.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2FyZSI7czo1OiJyZXNpZCI7czo4OiIyMC81LzczNSI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xNjIvNy8xMDAyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)