Abstract
Background: Medical advice columns in newspapers can provide a valuable service by educating the general public about important health-related issues. However, these columns may be harmful if the advice or information given in them is incomplete, inappropriate or misleading. The objective of this study was to assess the safety and appropriateness of advice given to elderly readers of newspapers in medical advice columns.
Methods: Medical advice columns published in Canadian newspapers in 1995 were identified from a CD-ROM database. The articles that were selected were published in English and contained medical advice pertinent to elderly people about topics that could be found in a textbook of geriatric medicine. Fifty articles, randomly selected from the 109 articles that met these criteria, were independently assessed by 5 geriatricians. A scoring system was used to rate the ability to determine to which population the article applied, how well fact was distinguished from opinion, the degree to which critical issues were addressed, the safety and the appropriateness of the advice. When the kappa statistic for inter-rater agreement was 0.74 or less, a 2-stage Delphi process was used in an attempt to reach consensus.
Results: Agreement (kappa > 0.74) was eventually achieved for 232 (92.8%) of the 250 ratings. In 4 (8%) of the articles there was a high probability that the advice given could be applied to the wrong patient population; in 7 (14%) there was a high probability that opinion might be interpreted as fact; and in 11 (22%) the major critical issues were not identified. Of greatest concern, however, the advice in 25 (50%) of the articles was judged to be inappropriate, and in 14 (28%) advice may have been dangerous and potentially life-threatening.
Interpretation: Although medical advice columns have the potential to improve the health of elderly readers, a significant percentage of these articles contain inappropriate or even potentially dangerous advice.
Health-related information is widely available to the public. [1] One forum for that information, the newspaper medical advice column, can provide a valuable service by educating the general public about important health-related issues. However, these columns can potentially harm readers if the advice or information provided is incomplete, inappropriate or misleading. This is a concern because medical advice columns attempt to deal with complex health issues in a brief format, they are read by a large heterogeneous audience, and they are not peer reviewed.
Studies [2, 3] have reported serious deficiencies in the general reporting of medical issues in newspapers; however, the appropriateness of medical advice given in newspaper columns has never been critically assessed. Our objective was to examine the safety and appropriateness of such advice based on critical review by a panel of specialists in geriatric medicine.
Methods
We reviewed 11 Canadian daily newspapers, each with a circulation over 100 000 and available in electronic format (Canadian NewsDisc CD-ROM); they contained weekly or monthly medical advice columns written by 7 different physicians. Only 2 of the physicians stated their specialty (internal medicine and family medicine). Articles appearing in 1995 newspapers were selected if they were published in English, contained medical advice pertinent to elderly people and the subject of interest could be found in a standard textbook of geriatric medicine.
A sample of 50 articles was randomly chosen from the 109 advice columns that were initially selected. The electronic format we used allowed us to delete any information that might identify the physician or the newspaper and allowed for a standard layout to further ensure rater blinding.
Initially, 5 specialists in geriatric medicine, certified by the Royal College of Physicians and Surgeons of Canada, independently scored the 50 articles. The scoring system, a modification of the index of scientific quality (ISQ), [4] included 5 ISQ categories we believe to be critical to the assessment of the safety and appropriateness of medical advice - raters' ability to determine to which population the advice applied, their ability to distinguish opinion from fact, the degree to which critical issues were identified, the safety and the appropriateness of the advice (see Appendix Table 3 for sample of the scoring sheet).
In the second stage of assessment, the raters met to evaluate individual columns using a process consistent with the Delphi process, a consensus-building technique. [5–7] For questions on which the kappa statistic for inter-rater agreement was 0.74 or below after the initial independent assessment, the group reviewed the rating a second time. Specifically, a facilitator (F.J.M.) showed the geriatricians the spread of scores for a particular rating and asked them to reread the article and independently rescore the item without discussion. The rescoring cards were visible only to the facilitator. The physicians were then informed of the new spread of scores. If the kappa statistic for a particular rating was greater than 0.74 the group moved on to the next rating; if it was still 0.74 or below a discussion limited to 5 minutes ensued. After the discussion the same blinded technique for rescoring was carried out, and the score that was determined at this point was considered final. The cutoff point of 0.74 for the kappa statistic was based on reports that a score between 0.6 and 0.8 signifies substantial inter-rater agreement. [8, 9]
Results
The target audience was clearly distinguishable (applicability) in only 35 (70%) of the 50 articles reviewed (Table 1). In how well fact could be differentiated from opinion, 42 (84%) articles were either potentially unclear or misleading. Critical issues were only partially covered or not covered at all in 38 (76%) of the articles, and total of 29 (58%) were found to provide unsafe or potentially dangerous advice. Specific examples of such advice are given in Table 2. The raters also felt that in half of the articles the advice provided was inappropriate.
After the first independent review, agreement (i.e., kappa score > 0.74) was reached on 96 (38.4%) of the 250 ratings. The second review resulted in agreement on 142 (56.8%) ratings, and the final discussion resulted in agreement on 232 (92.8%) of the 250 ratings (Table 1). The difficulty in achieving consensus was roughly equal for each of the 5 categories, in no single category was there consistent disagreement at various stages of the Delphi process.
Interpretation
Our results indicate that elderly readers following the advice provided in medical advice columns in Canadian newspapers could potentially be harmed. It is particularly concerning that a significant proportion of articles may be transmitting unsafe or potentially dangerous information. These results are consistent with previous evaluations of the quality of health and scientific coverage in the media that reported significant problems such as major factual errors and misleading information. [10–12]
Considering that medical journals and textbooks require several pages to delineate the salient issues regarding the diagnosis and management of certain conditions, it is not surprising that those writing much shorter advice columns have difficulty covering all of the important information in a way that is complete and understandable to the general public. Nevertheless, the accuracy and thoroughness of advice provided in these columns is critical if specific actions are recommended. Indeed, given that physicians are better able to assess the appropriateness of medical information than the general reader, it is ironic that articles submitted to scientific journals must undergo a peer review process before publication, but no such process exists for medical advice columns in newspapers.
Many of the advice columns we reviewed contained errors of omission rather than misstatements of fact. It is unclear whether these omissions were made by the authors or if they occurred during the editorial process; ideally, medical columns should be edited by people with some medical knowledge. It might be enlightening to examine how these columns are processed at different newspapers.
Several limitations of our study warrant discussion. First, our sample was limited to columns dealing with geriatric health concerns, although it would be surprising if problems similar to the ones we report do not exist with other medical advice columns. Second, the rating physicians are certified in geriatric medicine, hold academic affiliations and are qualified to make judgements regarding the appropriateness of advice, but we only surveyed the opinions of 5 physicians. Third, for a number of columns agreement among raters about the appropriateness of the advice was inconsistent until the last stage of the Delphi process. Nonetheless, the Delphi technique is a standard method for achieving group consensus in situations that require judgement or expert opinion, and we felt that, by incorporating the features of anonymity, iteration and feedback, it was the best way to arrive at a group consensus. Finally, we used selected portions of the ISQ, but because we did not attempt to compile individual ratings into an aggregate score for each column, this should not have posed a threat to the reliability of our method. Also, the individual items we selected have a high degree of face validity.
The relative quality and safety of medical advice provided in these columns might be improved in a variety of ways - with the use of a simple checklist similar to the scoring sheet we used, through a streamlined peer review process to highlight important issues not addressed by the columnist, or through feedback provided by physicians reading these columns, possibly in the form of a letter of the editor.
The physicians who write medical advice columns should be commended for attempting to educate readers about health-related issues. The support and assistance of the medical profession as a whole could greatly improve the quality of the information provided.
Competing interests: None declared.
This article has been peer reviewed.
Reprint requests to: Dr. Frank Molnar, The Ottawa Hospital - Civic Campus, Division of Geriatric Medicine, 1053 Carling Ave., Ottawa ON K1Y 4E9; fax 613 761-5334
References
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.