Efficacy of statins for primary prevention in people at low cardiovascular risk: a meta-analysis

We thank Dr. Tejani and colleagues for their comments, which highlight challenges associated with using data from meta-analysis to make clinical decisions in patients with low risk of future disease. Although some issues are not elaborated upon in the letter, we feel they deserve comment. First, available evidence suggests that individuals at differing levels of risk enjoy similar proportional benefits from the use of statins. Specifically, an individual patient data meta-analysis recently demonstrated similar relative risk reductions across a wide range of future cardiovascular risk.(1) We reported relative risks in the abstract because of the potential for absolute risk differences to mislead when event rates in control groups differ.(2) We hope that clinicians would individualize treatment for individual patients using relative risks, according to recommended practice.(3)

Dr. Tejani's letter includes an alternative interpretation of our estimate of the pooled risk of serious adverse events and withdrawals across treatment groups (RR 1.01 for statin recipients vs placebo, 95% CI 0.96-1.07; I2 = 8%) This interpretation assumes that efficacy endpoints, such as myocardial infarction and mortality, are included in the analysis of total serious adverse events observed. Our interpretation is based on a more conventional assumption - that efficacy endpoints are removed from analyses of serious adverse events. We based our assumption on firsthand experience with previous trials as well as supplementary trial data. For example, the FDA Medical Reviewer's report of JUPITER states "Any adjudicated cardiovascular death was classified as an efficacy endpoint and not an [adverse event]".(4)

We could not always determine whether efficacy and safety endpoints were analysed separately, based on available trial reports. It is also important to point out that our definition of serious events also included those events that led to withdrawal of assigned treatment, not only those that were life-threatening.However, we acknowledge that properly interpreting how harm was experienced, measured and reported by trialists is of special importance when achievable benefits are low - and that our claim of benefit is very sensitive to this assumption. We would be happy to supply the forest plot to Tejani and others upon request.

Lastly, we respectfully disagree with certain points in Tejani's letter. The Kyushu Lipid Intervention Study Group (KLIS) trial was in fact a randomized trial - and the trial met inclusion criteria for our systematic review. While we acknowledge there was compelling evidence that allocation to treatment groups was poorly conducted (a phenomenon which has been associated with exaggerated estimates of effectiveness) the analysts used statistical methods in an attempt to reduce systematic bias. We also disagree with Tejani's suggestion that a narrow confidence interval implies robust findings. A narrow confidence interval only implies reduction in random error. As we have already described (and Tejani suggests), a multitude of factors including trial conduct and measurement assumptions could contribute non-random error that potentially influence the conclusions of a study. We did not attempt to formally adjust for systematic or other sources of non-random error but would invite others to do so using existing approaches.(5)

Sincerely, Donald Husereau, Braden Manns, Anita Lloyd, Marcello Tonelli

1. Cholesterol Treatment Trialists'. Efficacy of cholesterol- lowering therapy in 18?686 people with diabetes in 14 randomised trials of statins: a meta-analysis. The Lancet. 2008 Jan 18;371(9607):117-25.

2. Cates CJ. Simpson's paradox and calculation of number needed to treat from meta-analysis. BMC Med Res Methodol. 2002;2:1.

3. Glasziou PP, Irwig LM. An evidence based approach to individualising treatment. BMJ. 1995 Nov 18;311(7016):1356-9.

4. Roberts MD. Center for Drug Evaluation and Research; Application number: 21-366/s-016, Medical Review(s), CRESTOR? (rosuvastatin calcium) [Internet]. 2010 Feb 5;Available from: http://www.accessdata.fda.gov/drugsatfda_docs/nda/2010/021366s016MedR.pdf

5. Maldonado G. Adjusting a relative-risk estimate for study imperfections. Journal of Epidemiology & Community Health. 2008 Jul 1;62(7):655-63.

Conflict of Interest:

As cited in article.

Tonelli et al's recent systematic review of statins for primary prevention [1] may mislead readers in a number of ways: 1) the review implies that the meta-analytic results are relevant to low-risk individuals, 2) the abstract does not present the benefits as absolute risk reductions, 3) the interpretation and conclusions in the abstract do not reflect the authors' assessment of the risk of bias, 4) it includes a non-randomized trial [2] in a pre-planned analysis of "only" randomized trials and 5) the abstract and conclusions ignore the total serious adverse event (SAE) outcome data . While each of these represents a serious flaw, we focus only on the SAE data, which we consider is the most important.

In the results section of the review Tonelli et al. write: "Twenty- one trials (n = 47 589) reported the number of participants who had serious adverse events. The trial-level relative risk could not be estimated for four trials that reported no events in either group. From the remaining 17 trials (n = 47 021), the pooled risk of serious adverse events did not differ significantly between treatment groups (RR 1.01, 95% CI 0.96-1.07; I2 = 8%)." While we congratulate the authors for obtaining SAE data from 21 of 29 trials, we urge them to post the Forest plot and raw SAE data from this analysis on CMAJ's website for use by other researchers. The pooled SAE relative risk confirms the finding of the UBC Therapeutics Initiative that total serious adverse events are not reduced by statins in the clinical setting of primary prevention. [3,4] Based on this finding the Therapeutics Letters concluded that statins do not have a proven net health benefit in primary prevention populations. The analysis by Tonelli et al., which furnishes SAE data from more trials, strengthens that conclusion.

What does the Tonelli et al. estimate of relative risk for SAE RR mean in clinical terms? The RR of 1.01 implies that taking a statin increases the chance of dying, being hospitalized or permanently disabled by a non-significant 1% as compared with placebo. The narrow confidence interval means that the finding is precise and robust (unlikely to be overturned by additional trials). The I2 of 8% means that the effect estimates from the 17 different trials are consistent and homogeneous. There is also a low risk of bias for this outcome as rigorous reporting of SAE is a regulatory requirement for all clinical trials. If anything, this may be an underestimate of the real RR, because the authors of the 8/29 trials that were missing the SAE outcome would not provide the SAE data A conservative assumption is that serious adverse events in those trials were more frequent in the statin group than the placebo group.

Why do Tonelli et al. ignore the SAE outcome in their conclusions? Perhaps the SAE outcome data create a dilemma for proponents of widespread statin use in primary prevention. How can statins cause a statistically significant reduction in all-cause mortality, non-fatal MI and non-fatal stroke, all outcomes that represent serious adverse events and are counted as such within the SAE data reported in clinical trials, and yet not reduce total serious adverse events? We can envisage two possible explanations for these findings. First, the observed reduction in mortality, non-fatal MI and non-fatal stroke reflect bias in the conduct of the RCTs and are not a real effect. [4] Tonelli et al recognize a moderate risk of bias in the RCTs making up this review. A second explanation is that statins cause an increase in some other serious adverse events that negate the expected reduction. Since the absolute risk reductions for the 3 SAE outcomes that are reduced are small, 0.3 to 0.7%, the absolute risk increases of other SAEs required to negate them would also be small. For informed patient choice, it is not essential to know the explanation. We consider that the key information primary prevention patients need to know is that taking a statin daily for 5 years does not reduce their chance of having a serious adverse event (dying, being hospitalized, or suffering permanent disability).

In summary, we consider that the most important finding in Tonelli et al's meta-analysis is that statins do not reduce total serious adverse events and thus do not provide a net health benefit in primary prevention populations. Because the authors failed to appreciate the importance of serious adverse events as an outcome, they draw the opposite conclusion that statins are beneficial and their use should be promoted in this population.

References

1. Tonelli M, Lloyd A, Clement F et al. Efficacy of statins for primary prevention in people at low cardiovascular risk: a meta-analysis. CMAJ 2011. DOI:10.1503/cmaj.101280

2. Pravastatin use and risk of coronary events and cerebral infarction in japanese men with moderate hypercholesterolemia: the Kyushu Lipid Intervention Study [KLIS]. J Atheroscler Thromb 2000; 7:110-21.

3. Therapeutics Initiative. Do statins have a role in primary prevention? An update. Therapeutics Letter 2010, Mar-Apr; 77:1-2 http://www.ti.ubc.ca/letter77 (accessed Dec 12 2011)

4. Therapeutics Initiative Do Statins have a Role in Primary Prevention? Therapeutics Letter 2003, Apr-Jun;48:1-2 http://www.ti.ubc.ca/newsletter/do-statins-have-role-primary-prevention (accessed Dec 13 2011).

Conflict of Interest:

None declared

I thank Dr. Quail for his letter.

In reply, I will pose an equally rhetorical question: when can authors expect that those writing letters to a journal would actually have read the article to which they refer?

On the first page of the "Interpretation" section, our article says: "In a meta-analysis of the efficacy and safety of statins among patients with coronary artery disease, the number needed to treat was 86 to prevent a single death from any cause and 62 to prevent a single nonfatal myocardial infarction. The corresponding numbers needed to treat among people at low cardiovascular risk by our primary definition were 239 and 153, which reflect the generally low rates of vascular events in this population. "

The statements in the abstract and the Interpretation section are both correct and I do not agree that either statement is dishonest.

Conflict of Interest:

An author of the article