Development and validation of the Osteoporosis Risk Assessment Instrument to facilitate selection of women for bone densitometry

Osteoporosis frequently results in fractures that lead to pain, deformity and disability. Wrist, spine and hip fractures are associated with substantial costs to the individual and to society.[1–3] Rates of osteoporotic fractures increase exponentially with age.[1, 4] Hip fracture rates are projected to double within 15 years[5, 6] and to increase almost fourfold by 2041.7 People at greatest risk of osteoporotic fractures are identified through the measurement of bone mineral density (BMD),[8–11] preferably by means of dual-energy x-ray absorptiometry (DXA).11 Although mass screening for osteoporosis is not recommended,[12–14] DXA testing in high-risk groups is essential to establish a diagnosis of osteoporosis. This may allow prophylactic treatment for the prevention of further bone degeneration and fracture.

DXA is the fastest growing single test in medicine. In Ontario the number of bone density tests increased from 34 402 in 1992 to 165 630 in 1997.15 At present there is no clear method for deciding who should undergo DXA testing16 because guidelines are not sufficiently precise to allow clinicians to decide which women warrant further testing.[8, 9, 11, 17] In effect, physicians may have little choice but to test all women around the time of menopause. The objective of this study was to develop and validate a simple screening tool to assist physicians in selecting patients for bone densitometry.

Methods

The Canadian Multicentre Osteoporosis Study (CaMos) is a 5-year cohort study evaluating the relation between risk factors for osteoporosis, measures of bone integrity and osteoporotic fracture.18 In brief, an age-, sex- and region-stratified random sample of the Canadian population was selected using a telephone-based sampling frame. Included in the CaMos population were noninstitutionalized people 25 years of age and older residing within 50 km of one of the 9 study centres (Vancouver, Calgary, Saskatoon, Hamilton, Toronto, Kingston, Quebec City, Halifax and St. John's). Native populations residing in the northern regions of the country were excluded. Participants had to be fluent in at least English or French or, in Toronto and Vancouver, Chinese. The collection of baseline data began in February 1996 and ended in September 1997. Eligible subjects were invited to meet with a trained interviewer to complete a standardized questionnaire and visit the clinic for bone densitometry. Data were collected on risk factors for osteoporosis, including age, race or ethnic background, personal and family history of bone fragility or fractures, medical history (e.g., comorbid conditions, reproductive history), current and past medication use, anthropometric information and lifestyle factors (dietary intake of calcium and vitamin D, alcohol consumption, smoking, physical activity and sunlight exposure).

For our study we used CaMos baseline data from the 3 Ontario centres. Cognitively normal women (Mini-Mental State score greater than 2019) who had undergone DXA testing at both the femoral neck and the lumbar spine (L1-L4) were eligible. We excluded those with a diagnosis of osteoporosis or taking bone active medications other than ovarian hormones (calcitonin, bisphosphonates or fluoride) at the time of study. About two-thirds of the eligible women were randomly allocated to the development of the assessment tool and the remainder to its validation.20 The preliminary analysis revealed that the prevalence of osteoporosis was less than 1% among women aged less than 45 years; therefore, we restricted our study population to women aged 45 or more.

BMD was measured using QDR 1000 DXA machines (Hologic, Inc., Bedford, Mass.) in Kingston and Toronto and a Lunar DPX Alpha machine (Lunar Corporation, Madison, Wis.) in Hamilton. To permit pooling of BMD measurements across study sites, CaMos standardized BMD values to Hologic equivalents.21 Hologic-equivalent BMD values were used by CaMos to determine the normal BMD values for young Canadian women (unpublished observations). These data served as the reference T score in our study. A low BMD at either the femoral neck or the lumbar spine is clinically relevant for deciding about prophylactic treatment to prevent osteoporosis and fragility fractures.11 Most patients with osteoporotic fractures have a BMD T score that is 2 standard deviations (SDs) below the mean for young adults.[17, 22] A value of 2 or more SDs below the mean has been referred to as the fracture threshold.[22, 23] In addition, the US National Osteoporosis Foundation promotes pharmacological therapy to reduce the risk of fracture among women with a BMD value more than 2 SDs below the mean for young normal adults.8 We chose a BMD value of 2 or more SDs below the mean for young Canadian women at either the femoral neck or the lumbar spine as the main outcome measure for our study.

Information on risk factors was obtained from responses to the CaMos questionnaire. Logistic regression analysis was used to evaluate the relation between each risk factor and a low BMD at the femoral neck and at the lumbar spine separately. Backward selection and stepwise approaches were used in model building.24 An effort was made to maximize predictive performance using variables that can be easily determined in clinical practice.[20, 25] Variables that best predicted low BMD of the femoral neck or lumbar spine separately were considered for inclusion in a model to predict low BMD at either of these 2 sites. Scores were assigned by rounding the odds ratio estimates to the nearest integer, assigning a score of zero to the reference group.[26–28] Receiver operating characteristic (ROC) curves were plotted for each model to determine the area under the ROC curve and the sensitivity, specificity and positive predictive value (PPV) at each threshold score.[29–31] Exact binomial confidence intervals were calculated. To ensure that few subjects with a BMD T score of 2 or more SDs below the mean would be missed, threshold scores for recommending testing with DXA were chosen to yield 90% sensitivity or greater. Selection of final variables for inclusion in the Osteoporosis Risk Assessment Instrument (ORAI) was based on comparison of the sensitivity, specificity and PPV when different numbers of clinical variables were included. Sensitivity analyses were performed by comparing the predictive ability of different models to identify the complement of normal BMD (T score more than 1 SD below the mean) and osteoporosis (T score of 2.5 or more SDs below the mean).9 Analysis of the development cohort was used to select the final criteria for inclusion in the ORAI. ROC analyses were then used to assess the discriminatory performance of the ORAI in the validation cohort.

Results

The Ontario sample of CaMos comprised 1930 women aged 45 or more years, of whom 1553 (80.5%) had undergone DXA testing at both the femoral neck and the lumbar spine. Of these, 9 were excluded because of cognitive impairment, and a further 168 were excluded because they had a diagnosis of osteoporosis or were taking bone active medications other than ovarian hormones. Of the remaining 1376 women, 926 were randomly allocated to the development of the tool and 450 to its validation. There were no significant differences in demographic characteristics between the development and validation cohorts (Table 1).

In bivariate analysis, age, race (white v. non-white), age at menarche, menopause, weight, sunlight exposure in past year, no alcohol consumption, previous minimal trauma fracture (wrist or forearm, hip, back, pelvis or rib) at age 45 or older, current estrogen use and current progesterone use were associated with low BMD at one or both sites (data not shown). After adjustment, age, weight and current estrogen use were common independent correlates of low BMD at both the femoral neck and at the lumbar spine (Table 2). Other significant correlates of low BMD varied between the 2 bone sites: current physical activity and previous minimal trauma fracture for the hip, and menopause for the lumbar spine. The 6 predictors of low BMD at either the femoral neck or the lumbar spine (age, weight, current estrogen use, menopause, physical activity, and previous minimal trauma fracture at age 45 or older) were considered for inclusion in the ORAI. The discriminatory performance of a scoring system with all 6 variables and of models with fewer variables (eliminated through backward selection) is presented in Table 3. The discriminatory performance of the one-variable model (weight < 70 kg) was significantly less than that of the model with at least 3 variables (weight, age and current estrogen use) (p < 0.05 for difference between areas under ROC curves). In fact, no threshold score using weight alone permitted 90% sensitivity. Models with at least 4 items (age, weight, current estrogen use and menopause) selected all women for DXA testing who were aged 65 years or more and not currently taking estrogen. Models with 3 and 2 items selected all women aged 65 years or more for DXA testing regardless of their estrogen use. Whereas the sensitivity, specificity and PPV of the 3-item instrument was similar to more complex models (those with 4 to 6 items), the specificity of the 2-item scoring algorithm was lower. As a result, 3 items (age, weight and current estrogen use) were included in the final model (i.e., the ORAI) (Table 4). A score of 9 or greater identified 90% of women with a BMD T score of 2 or more SDs below the mean and was therefore chosen as the threshold to recommend further testing with bone densitometry.

Table 5 provides summary statistics of the ORAI's performance in identifying women with the complement of normal BMD, low BMD and osteoporosis in the development and validation cohorts. The discriminatory performance of the ORAI did not differ significantly between the development and validation cohorts (areas under ROC curves 0.79 and 0.77 respectively, p > 0.05). At the recommended threshold score of 9, the 3-item ORAI had a sensitivity of 90.0%, a specificity of 45.1% and a PPV of 32.5% for identifying women with low BMD in the development cohort; the corresponding values in the validation cohort were 93.3%, 46.4% and 34.6%. In addition, the sensitivity was 97.0% in the development cohort and 94.4% in the validation cohort for selecting women with osteoporosis. With a T score of less than 1 SD below the mean, specificity increased to 56.8% and 58.2% in the development and validation cohorts respectively; in other words, the ORAI selected about 43% of those with normal BMD values for further testing with DXA. Overall, use of the ORAI represented a 38.7% reduction in DXA testing compared with screening all women in our study.

Interpretation

Screening all women for osteoporosis using bone densitometry is not recommended,[12–14] and even selective use has been criticized in "well women".32 However, practice guidelines recommend bone densitometry for the diagnosis of osteoporosis and for deciding on treatment.[8–10, 17] As a result, clinicians must use clinical factors to determine who is at increased risk for osteoporosis and should thus undergo further testing for low BMD. No clear consensus exists as to which factors clinicians should use to guide this decision.16 We found that a simple assessment using 3 factors that can be easily determined during a clinical interview successfully identified over 90% of women at increased risk for osteoporotic fractures. At the same time, fewer than half of those with normal BMD would be selected for testing.

The ORAI supports selective DXA testing in women aged 65 years or more, in women aged 45 years or more who weigh less than 60 kg and in women aged 55-64 years who weigh 60-70 kg and are not taking estrogen. Although selection of women aged 65 years or more is consistent with the US National Osteoporosis Foundation guidelines,8 the latter also recommend selection of postmenopausal women under 65 years of age who have one or more risk factors for osteoporosis other than menopause. The ORAI provides more specific recommendations for selection based on current weight and estrogen use; the result would be substantially less use of bone densitometry. For example, the selection of all women aged 65 years or more, in addition to post-menopausal women who currently smoke, weigh less than 127 lb (57.6 kg), have a history of fracture as an adult or have a history of fracture in a first-degree relative (the 4 risk factors highlighted by the National Osteoporosis Foundation), had a sensitivity of 92% but a specificity of only 21% for identifying low BMD in our sample.

Other approaches to selecting women for bone densitometry have been proposed.[33–35] Michaëlsson and colleagues33 suggested body weight as the sole criterion for osteoporosis screening. Our results also showed that weight less than 70 kg was the single best indicator of low BMD. However, because of low sensitivity, weight alone is insufficient to select women for bone densitometry. The Simple Calculated Osteoporosis Risk Estimation (SCORE), created by Lydick and colleagues,34 uses an index based on age, race, rheumatoid arthritis, history of nontraumatic fracture after age 45, estrogen use and weight. Although evaluations of SCORE in a Toronto study revealed good sensitivity, selecting 90% of women with low BMD, SCORE also selected 68% of women with a BMD value not more than 2 SDs below the mean.36 Finally, Weinstein and colleagues35 identified factors associated with osteoporosis among 1346 postmenopausal women referred for DXA testing. They suggested that post-menopausal women aged 61 years or more who weigh 165 lb (74.8 kg) or less be screened and that, among post-menopausal women who have never received oral estrogen therapy (oral contraceptives or hormone replacement therapy), either age or weight be used as the screening criterion. When applied to our postmenopausal women, the sensitivity was similar to that of the ORAI (89%); however, the specificity was lower (32%).

This is the largest Canadian study to provide a comprehensive evaluation of osteoporosis risk factors and "gold-standard" testing with DXA. However, the generalizability of the ORAI may be limited because of the low response rate (42%) in the CaMos Ontario sample. Further validation of the ORAI is required to evaluate its discriminatory performance using data from different populations and to explore results using current manufacturer norms.25 Although the ORAI may help to guide decisions about the need for bone densitometry, clinical judgement in individual cases is always important. The ORAI is not intended for use in women at high risk for secondary osteoporosis (e.g., long-term corticosteroid use and primary hyperparathyroidism). In addition, the ORAI does not address follow-up DXA testing for future diagnostic evaluation, or the follow-up efficacy of treatment strategies. Physicians can refer to published Canadian guidelines for recommendations about these issues.11 Furthermore, physicians should promote bone health independent of decisions about DXA testing, including physical activity, adequate nutrition (calcium and vitamin D in particular) and avoidance of tobacco and excessive alcohol.[8, 11]

Incidence rates of hip fracture are projected to double within the next 15 years if preventive measures are not taken.[5, 6] DXA testing for low BMD identifies which women are most in need of preventive treatment11 and significantly increases the use of hormone replacement therapy.[37, 38] The 3-item ORAI provides a simple method for making clinical decisions about the need for DXA testing. Targeting high-risk populations is important for achieving cost-effective interventions.39 In Ontario, DXA testing costs $81-$101, as compared with $41-$65 for a mammogram.40 Use of the ORAI may help to limit unnecessary costs. The ORAI identifies the majority of women at risk for osteoporosis, yet it limits the need for unnecessary testing among those with normal BMD.

We thank the Canadian Multicentre Osteoporosis Study (CaMos) site directors for permission to use their data for this study: Dr. Timothy Murray (Toronto site), Dr. Jonathan D. Adachi (Hamilton site), Dr. Tassos Anastassiades (Kingston site) and Dr. Alan Tenenhouse (CaMos principal investigator).

This study was supported by a grant from the University of Toronto Dean's Fund. CaMos was funded by the Seniors Independence Research Program through the National Health Research and Development Program (project no. 6605-4003-OS), the Medical Research Council of Canada (MRC), the MRC-PMAC (Pharmaceutical Manufacturers Association of Canada) Health Program, Merck Frosst Canada Inc., Eli Lilly Canada Inc., Procter and Gamble Pharmaceuticals Canada Inc. and Dairy Farmers of Canada. Dr. Susan Jaglal is a career scientist of the Ontario Ministry of Health; Dr. Jack Tu is supported by an MRC Scholar award.

This article was presented as a plenary poster at the meeting of the American Society for Bone and Mineral Group, Sept. 30-Oct. 1, 1999, in St. Louis, Mo.

Competing interests: None declared.