Table 2:

QUADAS16 assessment of the methodologic quality of included studies

StudySelection bias*Reference testDisease progression§Partial verification biasDifferential verification bias**Incorporation bias††Reference reviewer bias‡‡Index reviewer bias§§Clinical review bias¶¶Uninterpretable results***Withdrawals†††
Hoffman et al., 200010NoYesYesYesYesYesYesYesYesYesYes
Stiell et al., 20012YesYesYesNoNoYesYesYesYesNoYes
Stiell et al., 200313YesYesYesYesNoYesYesYesYesYesUnclear
Dickinson et al., 200426YesYesUnclearYesNoYesYesUnclearYesYesNA
Miller et al., 200619YesYesYesNoNoYesUnclearYesYesYesYes
Rethnam et al., 200820NoYesUnclearYesYesYesYesUnclearYesNoNA
Mahler et al., 200927NoYesYesYesYesUnclearUnclearUnclearYesYesYes
Stiell et al., 200921YesNoNoNoNoYesUnclearYesYesNoNo
Vaillancourt et al., 20092NoYesYesNoNoUnclearUnclearUnclearYesYesYes
Coffey et al., 201023YesYesYesNoNoUnclearUnclearUnclearYesYesYes
Stiell et al., 201024YesNoNo*NoNoYesUnclearUnclearYesYesYes
Duane et al., 201125YesYesYesYesYesYesUnclearYesYesYesYes
Duane et al., 201128UnclearYesUnclearYesYesYesUnclearUnclearYesYesYes
Griffith et al., 20117NoYesUnclearYesYesYesUnclearUnclearYesYesNA
Migliore et al., 201129NoYesUnclearNoNoUnclearUnclearYesYesNoNo
Inter-rater reliability, k0.540.000.15−0.130.210.470.390.17−0.03−0.020.00
Percentage agreement, %7387534753806753875333
  • Note: NA = not applicable.

  • * Was the spectrum of patients representative of the patients who will receive the test in practice? Is it a selective sample of patients?

  • Is the reference standard likely to classify the target condition correctly?

  • The 14-day proxy method was deemed to be an adequate reference standard because the outcome for all patients could be accounted for by either by the 14-day proxy method or radiography. This mirrors clinical practice.34 However, the 21-day surveillance strategy was deemed to be an inadequate reference standard because it assumes that patients with fractures missed at the initial presentation would be subsequently captured in patient logs. We found no data about the accuracy of the 21-day surveillance strategy to support its use as a reference standard.

  • § Is the time between the reference standard and the index test short enough to be reasonably sure that the target condition did not change between the 2 tests?

  • Did the whole sample, or a random selection of the sample, receive verification using a reference standard of diagnosis?

  • ** Did patients receive the same reference standard regardless of the index test result?

  • †† Was the reference standard independent of the index test (i.e., the index test did not form part of the reference standard)?

  • ‡‡ Were the reference standard results interpreted without knowledge of the results of the index test?

  • §§ Were the index test results interpreted without knowledge of the results of the reference standard?

  • ¶¶ Were the same clinical data available when the index test results were interpreted as would be available when the test is used in practice?

  • *** Were uninterpretable and/or intermediate test results reported?

  • ††† Were withdrawals from the study explained?