Table 2:

QUADAS (16) assessment of the methodologic quality of included studies

StudySelection bias*Reference testDisease progression§Partial verification biasDifferential verification bias**Incorporation bias††Reference reviewer bias‡‡Index reviewer bias§§Clinical review bias¶¶Uninterpretable results***Withdrawals†††
Hoffman et al., 2000 (10)NoYesYesYesYesYesYesYesYesYesYes
Stiell et al., 2001 (2)YesYesYesNoNoYesYesYesYesNoYes
Stiell et al., 2003 (13)YesYesYesYesNoYesYesYesYesYesUnclear
Dickinson et al., 2004 (26)YesYesUnclearYesNoYesYesUnclearYesYesNA
Miller et al., 2006 (19)YesYesYesNoNoYesUnclearYesYesYesYes
Rethnam et al., 2008 (20)NoYesUnclearYesYesYesYesUnclearYesNoNA
Mahler et al., 2009 (27)NoYesYesYesYesUnclearUnclearUnclearYesYesYes
Stiell et al., 2009 (21)YesNoNoNoNoYesUnclearYesYesNoNo
Vaillancourt et al., 2009 (2)NoYesYesNoNoUnclearUnclearUnclearYesYesYes
Coffey et al., 2010 (23)YesYesYesNoNoUnclearUnclearUnclearYesYesYes
Stiell et al., 2010 (24)YesNoNo*NoNoYesUnclearUnclearYesYesYes
Duane et al., 2011 (25)YesYesYesYesYesYesUnclearYesYesYesYes
Duane et al., 2011 (28)UnclearYesUnclearYesYesYesUnclearUnclearYesYesYes
Griffith et al., 2011 (7)NoYesUnclearYesYesYesUnclearUnclearYesYesNA
Migliore et al., 2011 (29)NoYesUnclearNoNoUnclearUnclearYesYesNoNo
Inter-rater reliability, k0.540.000.15−0.130.210.470.390.17−0.03−0.020.00
Percentage agreement, %7387534753806753875333
  • Note: NA = not applicable.

  • * Was the spectrum of patients representative of the patients who will receive the test in practice? Is it a selective sample of patients?

  • Is the reference standard likely to classify the target condition correctly?

  • The 14-day proxy method was deemed to be an adequate reference standard because the outcome for all patients could be accounted for by either by the 14-day proxy method or radiography. This mirrors clinical practice. (34) However, the 21-day surveillance strategy was deemed to be an inadequate reference standard because it assumes that patients with fractures missed at the initial presentation would be subsequently captured in patient logs. We found no data about the accuracy of the 21-day surveillance strategy to support its use as a reference standard.

  • § Is the time between the reference standard and the index test short enough to be reasonably sure that the target condition did not change between the 2 tests?

  • Did the whole sample, or a random selection of the sample, receive verification using a reference standard of diagnosis?

  • ** Did patients receive the same reference standard regardless of the index test result?

  • †† Was the reference standard independent of the index test (i.e., the index test did not form part of the reference standard)?

  • ‡‡ Were the reference standard results interpreted without knowledge of the results of the index test?

  • §§ Were the index test results interpreted without knowledge of the results of the reference standard?

  • ¶¶ Were the same clinical data available when the index test results were interpreted as would be available when the test is used in practice?

  • *** Were uninterpretable and/or intermediate test results reported?

  • ††† Were withdrawals from the study explained?