The validity of a sore throat score in family practice

In 1997 a national consensus conference on antibiotic resistance, sponsored by Health Canada and the Canadian Infectious Disease Society, recommended that antimicrobial prescriptions be reduced "by 25% within 3 years by focusing on community-acquired respiratory infection."1 The conference report noted that 80% of all antibiotic prescriptions were written by family doctors and suggested that specific guidelines be developed for the diagnosis and management of common infections.1

We previously developed a clinical approach to the evaluation of patients presenting with sore throat in family practice, suggesting that a 48% reduction in antibiotic use (relative to usual care) could be achieved with this method.2 However, mathematically derived prediction rules such as this can perform poorly when applied in new clinical settings, and validation in additional populations is recommended.3^,4^,5 The purpose of this study was to evaluate the performance of the score approach in a population of family medicine patients undergoing routine clinical care.

Methods

Ethical approval for this study was obtained from the University of Toronto.

The College of Family Physicians of Canada (CFPC) maintains a register of community-based family physicians who have participated in research studies (National Research System [NaReS]).6 In fall 1998, information about our study was mailed to NaReS physicians as well as to a random sample of physicians from the general membership list of the CFPC; all recipients were residing in Ontario.

Physicians approached patients 3 years of age and older (or their parents) who presented with a new infection of the upper respiratory tract and a sore throat. Children who were not yet talking were eligible for inclusion if they displayed signs of a new illness of the upper respiratory tract. Patients were ineligible for the study if they had used antibiotics in the preceding week, were immunocompromised or could not read English. Consent was obtained for participation in the study and for a throat swab to be taken. For children less than 16 years of age, consent was obtained from the parents.

For each patient, the physician completed a brief assessment form and obtained a throat swab. The swab was submitted to the physician's local laboratory for culture, and a copy of the result was forwarded to the study centre by the local laboratory through arrangement with the Ontario Association of Medical Laboratories. Physicians indicated on the assessment form whether or not they had prescribed an antibiotic and if they felt that the throat swab was warranted.

As part of a separate trial, all of the physicians were given details about the score approach. To determine the score,2 the physician assigns one point for each of the following: history of or measured temperature greater than or equal to 38∘C, absence of cough, tender anterior cervical adenopathy, tonsillar swelling or exudate, and age less than 15 years. One point is subtracted if the person is 45 years of age or older. If the total score is 1 or less, antibiotic therapy and culture of throat swab are not recommended. If the total score is 2 or 3, culture of a throat swab is recommended, and a decision about antibiotics should be based on the culture results. Patients with a score of 4 or more have the highest likelihood of disease, and either initiating treatment with an antibiotic or taking a throat swab for culture is appropriate.

For the present study, the main outcomes were the sensitivity and specificity of the score approach in community-based populations; these values were compared with published values from an academic family medicine centre.2 Sensitivity was determined from the number of patients for whom the score recommendation was to prescribe an antibiotic or take a throat swab for culture (those who "tested positive") and the number who had a group A streptococcal infection as indicated by a positive culture result ("true positives"). Specificity was determined from the number of patients for whom the score did not indicate antibiotics or a throat swab (those who "tested negative") and the number in whom culture results were negative ("true negatives"). In addition, if physicians prescribed an antibiotic but indicated that they would have taken a throat swab for culture, they were also considered to have appropriately managed the negative culture result.7 All patients with a score of 4 or more were considered to have received an antibiotic prescription and not to have undergone a throat swab. This provides the most conservative estimate of score performance.

A 𝛘² test was used to compare the sensitivity and specificity of the score approach in this patient population with the sensitivity and specificity observed in the original study.2 In addition, management by the community-based physicians, in terms of prescriptions for antibiotics and recommendations concerning throat culture, was compared with the management recommendations generated by the score approach.

Results

A total of 97 family physicians from 49 Ontario communities, representing 59.1% of the 164 physicians contacted, participated in the study. There were no differences in age or sex between physicians who participated and those who did not. Eighty-six physicians returned surveys providing information on themselves and their practice characteristiics. Family physicians in the study practised in communities with populations ranging from less than 10 000 (16 [19%] of 84 physicians) to greater than 100 000 (46 [55%] of 84 physicians). Fifty-nine of 81 physicians (73%) were in group practices and 50 of 63 physicians (79%) were reimbursed on a fee-for-service basis.

In total, 692 children and adults were assessed. Of these, 71 (10.3%) were excluded because other conditions were diagnosed (specifically, bronchitis, sinusitis, otitis media or pneumonia). Two-thirds of the remaining patients were female, and one-quarter were less than 15 years of age (Table 1). Most (611 [98.4%]) were seen between October and March, and more than half (330/585 [56.4%]) had been sick for 3 or fewer days. Almost all had a sore throat (579/621 [93.2%]), although relatively few 120 (19.3%) had only a sore throat and no other symptoms. The most common diagnoses recorded by the family physician were upper respiratory tract infection (294/618 [47.6%]), pharyngitis (128/618 [20.7%]), strep throat (46/618 [7.4%]) and tonsillitis (31/618 [5.0%]). Physicians prescribed an antibiotic to 173 (28.0%) of the 619 patients for whom prescription status was known. Of these 173 prescriptions, 109 (63.0%) were given to patients whose culture results were negative for group A Streptococcus.

The prevalence of group A Streptococcus in the cultured throat samples was 17.0% (102/600). The prevalence was higher among children (55/158 [34.8%]) than among adults (47/441 [10.7%]) (p < 0.001). According to the culture results, the prevalence of group A Streptococcus was 2/179 (1%) among patients with a score of 0 or -1, 10% (13/134) among those with a score of +1, 17% (18/109) for those with a score of +2, 35% (28/81) for those with a score of +3 and 51% (39/77) for those with a score of 4 or more. The corresponding likelihood ratios were 0.05, 0.52, 0.95, 2.54 and 4.93 respectively.

Overall, the sensitivity of the score approach for identifying group A Streptococcus infection was 85.0% (95% confidence interval 76.5%-91.4%) and the specificity was 92.1% (95% confidence interval 89.3%-94.3%) (Table 2). There was no difference in the sensitivity or specificity of the score between this population of patients and the patients seen in an academic family medicine unit.2 There were also no differences between these two populations when children and adults were compared separately.

Our original study found that the score approach was more sensitive than usual physician judgement for children.2 There was some variability between the two studies in the sensitivity of physician judgement for children (70.6% for the academic family medicine unit2 and 85.2% in this study, p = 0.10), although the prevalence of group A streptococcal infection in children was similar (36.2% and 34.8% respectively). When the results of the 2 studies were combined, physicians identified 70 (80%) of 88 group A streptococcal infections in children; the score approach identified 81 (94%) of 86 infections (p = 0.006 for difference between physician judgement and score approach).

Compared with usual physician care, management according to the sore throat score would have resulted in a 52.3% reduction in antibiotic prescriptions, a 63.7% reduction in unnecessary antibiotic prescriptions and a 35.8% reduction in the culture of throat samples (Table 3, all p < 0.01). The greatest reduction would have been in terms of unnecessary antibiotic use in adults. For children, there were no significant differences in terms of unnecessary prescriptions or overall antibiotic use, although use of laboratory testing would have been reduced.

Interpretation

A clinical score is accurate and reliable for determining the appropriate management of children and adults presenting to family physicians with an infection of the upper respiratory tract and sore throat. This approach could substantially reduce unnecessary prescribing of antibiotics by family physicians and is consistent with national recommendations for limiting antibiotic resistance.1

Standards for the evaluation of prediction rules recommend prospective validation,3^,4^,5 because these rules may not perform well in a clinical population if the prevalence of disease in the clinical population differs significantly from that in study populations.8^,9 Sore throat prediction rules in particular suffer from this problem.10 The prevalence of infection with group A Streptococcus in this community-based study was 17.0%, which is within the range of 10% to 20% found in most general practice settings.11^,12^,13 Therefore, the clinical score approach is probably applicable in most family physician offices in Canada.

Another concern may be that some cases of infection with group A Streptococcus are missed by the score approach. However, this concern should not be limited to the score approach. Physicians do not currently obtain a throat swab for every case of sore throat.14^,15^,16^,17^,18 The sensitivity of clinical judgement ranges from 50% to 75%,11^,12^,13^,19^,20 so physicians currently miss one-quarter to one-half of these infections. In addition, fewer than 15% of people with upper respiratory tract infection or pharyngitis seek medical care,21^,22^,23^,24 so it is likely that a significant number of infections with group A Streptococcus never come to the attention of a physician and are also missed.

Despite these problems, levels of rheumatic fever remain low in developed countries.25^,26^,27 As the main reason for treating pharyngitis caused by group A Streptococcus is to prevent rheumatic fever,7^,28^,29 this suggests that rates of rheumatic fever are unlikely to increase as long as the sensitivity of diagnosis is maintained at current levels. The clinical score approach is no less sensitive than usual physician care and thus is unlikely to adversely affect rates of rheumatic fever.

The score approach does not reduce unnecessary antibiotic use in children. However, in this age group it is more sensitive than usual physician judgement for identifying infection with group A Streptococcus. Diagnosis by physicians might have been somewhat more sensitive in this study than previously reported2^,11^,12^,13^,19 because the physicians were given information about the score approach as part of another study. However, their diagnostic performance in this and the preceding study2 was more variable than score recommendations. A high sensitivity for the diagnosis of group A Streptococcus in children is recommended7^,28^,29 and may be desirable, given that rheumatic fever still occurs more often in this age group.30^,31

Guidelines for the management of pharyngitis recommend the use of throat cultures because clinical diagnosis is inaccurate.7^,28^,29 Although antibiotic prescribing could probably be reduced through greater use of cultures, physicians have resisted this strategy.13^,14^,15^,16 Use of the sore throat score would retain the selective approach preferred by family physicians,14^,15^,16^,18 while minimizing the need for additional tests. This approach is valid and reliable and could help to reduce unnecessary antibiotic use in family practice settings.