Canada's programs to prevent mental health problems in children: the research–practice gap

The increased number of prevention activities directed at reducing child mental health problems in Canada is welcome. However, practitioners and policy-makers should reflect on 2 questions that were posed 20 years ago by Michael Rutter: “How much do we really know about prevention in the field of psychosocial disorders of childhood?” and “Is it necessarily the helpful enterprise that it is usually portrayed to be?”1

In this commentary we highlight examples of the mismatch between what is known and what is practised in the prevention of child mental health problems in Canada, identify some of the underlying factors that may contribute to the gap between research and practice, and make recommendations for strengthening the link between research and practice.

A number of preventive interventions have been shown in replicated randomized controlled trials (RCTs) to be effective in reducing child psychosocial problems. We briefly outline 3 exemplary prevention programs in the US and contrast these with a set of programs used in Canada.

The Nurse–Family Partnership. This is one of the most rigorously evaluated early-years interventions. It is an intensive US program of pre- and postnatal home visits by nurses to at-risk, first-time mothers. In 3 different RCTs the program has demonstrated multiple positive maternal and child outcomes in the short and long term. These include reductions in child abuse and neglect, in antisocial behaviour in youth and in behavioural problems related to the use of drugs and alcohol among adolescents.2^,3^,4 An equivalent program has not been implemented in Canada. A number of Canadian programs use the Healthy Families America model, in which home visitation is provided by trained lay staff. The home visitation component of the Healthy Babies, Healthy Children Program in Ontario, perhaps the largest in Canada, relies heavily on lay home visitors complemented by nursing visits and supervision. Unfortunately, it received no controlled evaluation prior to its province-wide dissemination.

The Healthy Families America program has shown limited effectiveness in RCTs, despite promising results under weaker research designs.5 Similarly, Olds and colleagues' recent RCT has shown that lay home visitors produced few significant improvements compared with a control group, while several positive differences were found in participants who received visits by nurses.4 Despite this, positive results of the Nurse–Family Partnership are often used to justify a variety of other home visitation programs. This is akin to suggesting that if one antibiotic is shown to be effective, another type within a related class can be assumed to work — regardless of duration, dosage or underlying mechanism.

The Incredible Years Program. Community-based group parenting programs are another important strategy aimed at the early years of child development. The Incredible Years Program has been shown to improve parenting practices, parent–child interactions and child behaviour through multiple, independently replicated RCTs.6^,7^,8 Nevertheless, there are few offerings of this program in Canada. However, several other parenting programs are actively promoted; for example, Nobody's Perfect has been widely disseminated throughout Canada. According to Health Canada, “Nobody's Perfect has proven to be an effective program for parents who have limited access to sources of help and information.”9 In fact, the evidence for program effectiveness is inconclusive. The most rigorous evaluation we have found is a pre/post design with no control group or standardized measures.10

The LifeSkills Training Program. For older children, this is arguably the most effective school-based substance abuse prevention program. It has been shown through replicated RCTs to reduce alcohol use, smoking rates and multiple substance abuse among young people.11^,12^,13 Despite the rigour of the evidence, we are unaware of any implementations of this program in Canada. However, many communities in Canada offer the Drug Abuse Resistance Education program (DARE), the most widely disseminated program of its kind in the United States.14 Unfortunately, there are serious concerns about the effectiveness of the DARE program, in which effect sizes for the reduction of substance abuse hover around zero.15

These examples illustrate the research–practice gap that has developed in Canada's prevention programs for child mental health problems. Although many factors are likely to have contributed to this gap, one of the most important of these is probably the strong desire to implement programs that are not resource-intensive. Yet most proven prevention programs are intensive and are provided over a relatively long period. As well, additional reinforcement may be required for already intensive programs (e.g., booster sessions, additional components, supports to reduce barriers to participation). These critical aspects increase costs and a need for long-term commitment.

Watering down effective approaches by using interveners with less training, reducing the duration and intensity of interventions, minimally investing in outreach and failing to provide support to maintain participation reduces costs. Some policy-makers appear to believe that “a little is better than nothing,” when in fact “a little” may result in no impact and no long-term savings. In contrast, more intensive and expensive evidence-based programs such as the Nurse– Family Partnership may result in long-term cost savings.16

Second, the fact that psychosocial prevention programs can do harm is rarely considered. A meta-analysis of randomized trials of juvenile awareness programs, such as “Scared Straight,” that aim to reduce juvenile delinquency by having at-risk youth meet with convicted felons, has found an increased rate of re-offending in those exposed to the intervention.17 Similar results have been found in other programs for at-risk youth.18^,19 The failure to recognize this potential may decrease the pressure for rigorous evaluation.

Third, some agencies target psychosocial problems for which there are no proven-effective interventions. As one example among many, no prevention programs have been shown to reduce suicide rates among youth.20

Recommendations

1. Systematic efforts to search the empirical literature are needed to identify effective programs before launching a prevention effort. Although this would appear to be self-evident, present community practice suggests that this approach is not routine. University–community collaborations may be of help. The development of a technical assistance program for community-based organizations is another possibility.21

2. New evaluations of effective programs may be required in some circumstances, if the original model is substantially altered or if the program is applied to a population that substantially differs from the one for which it was proven effective. It should be noted that all 3 programs highlighted earlier have demonstrated effectiveness in minority populations.2^,13^,22

3. Innovation is needed where there are no evidence-based programs. However, rigorous evaluation should be a mandatory part of the effort. There are precedents for combining community-developed programs with high-quality evaluations.23 In addition, dissemination of a novel intervention should not occur until there is a clear indication of whether the program provides more benefit than harm.

4. Current programs that have not been properly evaluated should not be disseminated. Although it may be too disruptive to discontinue all such programs until they are properly evaluated, at a minimum, further dissemination should be halted and specific investment in adequate evaluation undertaken. Programs found to have minimal positive impact need not be automatically abandoned. Enhancement and re-evaluation is an alternative approach. This is being attempted with some DARE programs. Such an approach may help reduce resistance to rigorous evaluation.

5. Strategic planning around prevention programming is urgently needed. The present haphazard approach to psychosocial prevention planning for children is inadequate. Effective prevention programs will likely require substantial resources for implementation, the intervention, rigorous evaluation and replicated evaluation. Substantial leadership and extensive partnerships across multiple government levels, universities and communities are required. The National Cross-Site Evaluation of High-Risk Youth Programs in the United States may provide part of the model for such a partnership.23 This government-sponsored effort required rigorous evaluation of outcomes of a variety of university and community-agency developed models for reducing substance abuse.

Although gains have clearly been made in developing a handful of evidence-based programs, few strides appear to have been made in Canada in reaping the benefits of this empirical work. There is promise in the prevention “enterprise” but much work is needed before it will be realized in Canada.