Taking risks with injury prevention

At SMARTRISK, we want you to take risks.

Don't forget your helmet, or your parachute. It's that simple. — SMARTRISK home page1

It is widely accepted that new public health initiatives should be scrutinized, just as we scrutinize the evidence on new drugs. Ideally, such programs should be evaluated before or soon after they are introduced, especially if they take a novel and untested approach or raise concerns about possible harm. This principle applies to messages about safety no less than to any other kind of public health message.

Some safety groups do take self-assessment seriously, and have published evaluations of the effectiveness of their programs.2^,3^,4^,5 Notably, the results are not all positive. Others6^,7^,8 have chosen to ignore this obligation. This ought to concern us; the public might not question or even consider the evidence base of safety messages delivered under the aegis of a nonprofit public-interest organization. When messages are perceived to be well meaning, they may also be perceived as well founded. But this is not necessarily the case.

Among such unexamined campaigns is the SMARTRISK program to encourage risk-taking while “preventing injuries and saving lives.”1 For a safety organization to actively promote risky behaviour ought to make us raise our eyebrows in surprise, if not downright alarm: after all, risk-taking is to injury occurrence as smooching is to the spread of infectious mononucleosis. Moreover, there is scant evidence that just telling people to take risks in a certain way actually influences their behaviour.9^,10^,11^,12^,13 Hence, to promulgate the message that risks can be taken “smartly” is a risky business indeed, and cries out for formal evaluation.

SMARTRISK's educational programs are intended to change the way young people assess risks: “At SMARTRISK, we believe that risk-taking is fun — something we want to be able to keep doing day after day, but we all have a line we should not cross … we call it the Stupid Line.”1 This confusing and potentially dangerous message is aimed at 15- to 24-year-olds, among whom the leading causes of both fatal and nonfatal unintentional injuries are motor vehicle crashes, falls and ingestions.14^,15 To reach this age group, the program uses “creative messages” to develop “a positive approach.”

The showpiece of the program, however, is “Heroes.” This multimedia presentation aimed chiefly at high-school audiences is the most compelling reason why evaluation is so urgently needed. “Heroes” presentations may feature an unsuccessful risk-taker, e.g., an adolescent who became quadriplegic in a crash when he was driving drunk. For the sound-and-light show packaging of the “take risks smartly” message, schools are charged $2500 for the first day and $1500 for each subsequent day of presentations at the same venue.1

Compounding the mixed message of “safe” risk-taking are semantic questions. Why is a survivor of a “stupid risk” a hero? Does a willingness to share one's misfortune confer heroic status? Do we know how adolescents respond to the label of “hero”?

And where is the evidence that teenagers can be taught to take risks “smartly”? In fact, the evidence suggests that relying entirely on education in risk reduction may actually be harmful.16^,17 For example, when some American states offered licencing at a younger age to those who completed a driver training program, the result was an increase in crashes among young drivers, presumably because this education was insufficient to offset the risks of youthfulness. Nevertheless, education-only prevention programs remain popular, deflecting energy and resources from attempts to promote safety in other, more effective ways. For example, if the Product Safety Branch of Health Canada simply issues advisories to pediatricians instead of banning dangerous products, injuries will continue to occur. Systematic reviews18^,19 have shown that effective prevention programs must include elements that go beyond education, no matter how smartly the messages are packaged. Changing behaviour is a complex and notoriously difficult task in this age group.15 Moreover, relying on education places responsibility on the victims (or their parents), who are then blamed if an injury occurs.

The lack of evaluation of the SMARTRISK program is difficult to comprehend not only from an academic standpoint, but also in view of the fact that, unlike many such nonprofit organizations, SMARTRISK received a $5 million dollar grant in 1999 from the Ontario Ministry of Health and Long-Term Care. This sizeable allocation appears to have been awarded without demonstration of the merit of the SMARTRISK strategy. The investment was described as “an opportunity to build and develop Ontario's injury prevention practices through program support, information sharing and development, education, and social marketing.”16 However, the grant was made without peer review and without reference to specific performance criteria.

Because of my concern about SMARTRISK's message, I wrote to the ministry requesting more information about the conditions of the grant. The reply was so uninformative that I eventually filed a Freedom of Information inquiry; several months later, and only after I appealed to the Freedom of Information Commissioner, I received a copy of the original agreement. I was amazed to discover that this grant in support of a program that differs radically from what most experts recommend rested on nothing more substantial than a 2-page memorandum of understanding. Surely the onus was on the ministry to seek some assurance that the program was both safe and effective.

If SMARTRISK were moving through growing pains or lacked funding, a lack of self-evaluation might be more understandable. But this is not the case. Moreover, it talks about the importance of research but seems not to acknowledge evaluation as a branch of research. Thus, by claiming research as a central concern, it takes a stance that amounts to a double standard. SMARTRISK president and CEO, Robert Conn, writes that “What is needed is an ability to sort the good research from the bad,” and proposes that his organization do this sorting.20 But in applying grant money to this effort, the program duplicates what has become a minor industry of systematic reviews and Cochrane collaborations. Then-Minister of Health Elizabeth Witmer announced in 1999 that “The key focus [of the grant] will be on research used to develop new programs and initiatives, including the development of a broad evaluation framework, a provincial research agenda, [and] attitudinal research on risk perception and risk reduction.”21 Nearly 3 years later, there is no sign that any of these goals have been achieved, the only product remotely along such lines being a commissioned report on the cost of injury.22

The burden of proof that the Heroes program works rests on its proponents' shoulders. Government funders, on behalf of taxpayers, should demand such proof: Web site testimonials are no substitute for rigorous scrutiny. The public needs to know whether the SMARTRISK program represents public money well spent. Even more, we need to know if the SMARTRISK approach will save lives or cost them.