Regression discontinuity designs: an approach to the evaluation of treatment efficacy in primary care using observational data

Example: prescription of statins in UK primary care

Statins are one of the most widely prescribed classes of drugs in UK primary care, accounting for more than £800m (€500m; $664m) of National Health Service expenditure in 2011.14 The effect of statins in primary prevention of cardiovascular disease has been widely investigated in randomised controlled trials and observational studies.15 16

NICE guidance on the use of statins for the primary prevention of cardiovascular disease suggests that statins should be prescribed to people whose risk of experiencing a cardiovascular event in the subsequent 10 years exceeds 20% (revised downwards to a 10% risk score threshold in July 2014), as calculated using an appropriate cardiovascular risk indicator (for example, the Framingham risk calculator at www.framinghamheartstudy.org/risk-functions/cardiovascular-disease/10-year-risk.php.17 Hence, the RDD could be used to estimate the effect of statins on low density lipoprotein cholesterol concentration for people around this 20% threshold under the assumption that those “just below” and “just above” the threshold are similar. If this assumption is valid, then we have two groups of treated and untreated individuals whom we consider to belong to the same population. In this example, the assignment variable is the 10 year cardiovascular risk score, the outcome variable is the low density lipoprotein cholesterol concentration, and the threshold is a 10 year cardiovascular risk score of 20%. Figure 1⇓ illustrates some example plots of low density lipoprotein cholesterol concentration against cardiovascular risk score. In plots A, B, and C, patients receive treatment only when they lie above the 20% risk score threshold. In plot D, for some patients (represented by the square icons), treatment is contrary to that inferred from their risk scores.

Fig 1 Graphical representations of various regression discontinuity design (RDD) scenarios using simulated data with 10 year cardiovascular risk score as assignment variable and low density lipoprotein (LDL) cholesterol concentration as outcome variable. d represents arbitrary maximum distance from 20% cardiovascular risk score threshold at which data points would be included in associated RDD analysis

• Download figure
• Open in new tab
• Download powerpoint

Figure 1⇑ panels A-C show different approaches to the analysis of these measurements, when treatment is applied strictly according to the guidelines. A “naïve” approach to estimating the treatment effect would be to take the difference in sample mean low density lipoprotein cholesterol concentrations between people above and people below the threshold, within a particular distance, d, either side of the threshold (fig 1⇑, panel A). The distance, d, defines the population to which the RDD is applied and quantifies the notion of similar people being just below and just above the threshold.

This naïve approach to estimation depends on the chosen value of d. A more appropriate estimate of the treatment effect at the threshold would be obtained by fitting linear regression models on either side of the threshold,18 because one might assume a linear relation between risk score and low density lipoprotein cholesterol concentration. If a constant relation between risk score and low density lipoprotein cholesterol concentration is assumed, then two separate linear regression models with the same slope would be fitted, one on each side of the threshold (fig 1⇑, panel B). Then, the difference in fitted low density lipoprotein cholesterol concentrations at the 20% threshold, estimated using these two models, would be an estimate of the effect of statins on low density lipoprotein cholesterol concentration. If the slopes cannot be considered the same below and above the threshold, as in figure 1⇑ panel C, then linear regression models with different slopes should be fitted on each side of the threshold or, perhaps, more flexible models could be considered.19

Figure 1⇑ panels A-C represent situations in which guidelines are strictly adhered to. In this context, an RDD is termed “sharp.” Many situations may exist in which guidelines are not interpreted strictly. For statin prescriptions, individual general practitioners may prescribe statins by using other information about patients, or patients might request a prescription for a statin irrespective of their risk score. In such scenarios, there will be people below the threshold who receive the treatment and people above the threshold who do not. In this case, the RDD is termed “fuzzy.” Figure 1⇑ panel D shows an example plot from a fuzzy RDD, where the square icons represent those patients who received treatment contrary to that indicated by their 10 year cardiovascular risk score.

Generally, most RDDs in clinical settings will be fuzzy and must account for the possibility that the threshold does not correspond to a strict separation between treated and untreated patients. A fuzzy RDD is analogous to a randomised controlled trial with partial compliance: there are patients allocated to a treatment arm (equivalent to being above the threshold in an RDD) who have not received a treatment and patients allocated to the control arm (below the threshold in an RDD) who have received a treatment. These patients are unlikely to represent a random sample in either a randomised controlled trial or an RDD setting. This problem can be alleviated by using a method for estimation of treatment effect that accounts for whether or not a patient actually receives the treatment above or below the threshold. This estimate is termed a “local average treatment effect,”19 20 and we view this as the treatment effect estimate from a sharp RDD but re-weighted to account for the fuzziness in the RDD. This method may be unreliable in some cases, typically where the threshold is not strongly associated with the treatment received. In this scenario, an RDD may not be appropriate for the estimation of a treatment effect.

Our estimates of treatment effect depend on the threshold being a reliable indicator of patients’ treatment. Figure 2⇓ provides some guidance on appropriate data plots to consider when attempting to use an RDD. Such plots can be helpful in exploring whether the threshold defines an acceptable separation between treated and untreated patients. In the statins example, if we plot risk score against low density lipoprotein cholesterol concentration and observe an obvious “jump” or discontinuity at the threshold level, this indicates that the threshold defines an appropriate point at which an RDD might be used.

Fig 2 Useful exploratory plots of assignment variable against outcome variable with threshold marked by the solid line. Panel A shows obvious sharp discontinuity at threshold where regression discontinuity design (RDD) could be used. B shows fuzzy scenario in which RDD could be used with some accounting for fuzziness. C shows no obvious discontinuity at threshold; one would not naturally consider RDD here

• Download figure
• Open in new tab
• Download powerpoint