The importance of study design in the assessment of nonnutritive sweeteners and cardiometabolic health

Although we applaud Azad and colleagues for a comprehensive review of nonnutritive sweeteners (NNS) and cardiometabolic health,1 we are concerned that important methodological considerations were overlooked.

In interpreting the pooled analyses of randomized controlled trials (RCTs), the authors did not account for the nature of the comparator.

One would expect that the intended effect of NNS would differ depending on the amount of energy that is available to be displaced from that comparator. Because the comparator in the included RCTs was water (4 of 7),2^–5 placebo (2 of 7)6^,7 or a matched weight loss diet (1 of 7),8 the design did not allow for energy displacement by NNS. Therefore, the NNS in these comparisons would not be expected to result in any meaningful weight loss or attendant cardiometabolic benefits, an expectation supported by the lack of effect observed in the pooled estimates for body weight, body mass index, insulin resistance and glycosylated hemoglobin.1

On the other hand, if one focuses on those RCTs in which NNS were used to displace excess energy from sugar-sweetened beverages as the comparator, then the results do favour the intended benefits of NNS. The RCT conducted by Maersk and colleagues3 was the only eligible RCT to include a direct comparison with sugar-sweetened beverages, a comparison Azad and colleagues1 elected not to use in their pooled analyses. Although NNS in displacing excess energy from sugar-sweetened beverages did not show a significant reduction in body weight change, this comparison did show significant reductions in liver fat, systolic blood pressure, triglycerides and total cholesterol.3

Therefore, the conclusion by Azad and colleagues1 that the evidence does not support the intended benefits seems unjustified.

The authors also accorded too much weight to prospective cohort studies in their conclusions. Although prospective cohort studies offer the best protection against bias among observational studies, based on their long longitudinal follow-up, ability to adjust for multiple confounding factors and ascertainment of clinical outcomes rather than surrogate biomarkers, residual confounding in these studies cannot be ruled out. In particular, it is well documented that prospective cohort studies of NNS are at a high risk of reverse causality, because higher consumers of NNS may choose these products because they are at a higher risk for adverse cardiometabolic outcomes and not the other way around.9 The prospective design of these studies does not necessarily limit this type of confounding, despite the authors’ claims to the contrary.

The evidence presented by the authors is also subject to other important limitations including publication bias (diabetes); inconsistency (diabetes and hypertension); indirectness (hypertension, stroke, coronary heart disease and cardiovascular disease); and imprecision in the pooled estimates (diabetes, hypertension, stroke and coronary heart disease).

These uncertainties suggest that the current estimates from prospective cohort studies are not trustworthy and new research is likely to have an important impact on their direction, magnitude and precision.

Taken together, the available evidence supports the intended benefits of NNS as being similar to that of other interventions to reduce energy intake, such as water. Although prospective cohort studies have shown an adverse association with incident hypertension, stroke and cardiovascular disease, these associations are discordant with higher-quality evidence from RCTs and are at high risk of reverse causality, among other important limitations. To improve the estimates, we agree that there is a need for more long-term RCTs of interventions using NNS to reduce excess energy from sugars, especially from sugar-sweetened beverages.