In theory, “evidence based policy making” should work well. Scientists produce evidence, which policy makers then use for decisions. In return, policy makers provide scientists with evidence requirements and resources for research. The approach has an intuitive, common sense logic.^1,2

In practice, there are problems—evidence based policy does not always work.^3,4 For example, it has been reported that many scientists are sceptical about the extent to which research is used, and that many policy makers are sceptical about the usefulness of research.⁵ This is because scientists and policy makers have different mentalities: for example, their goals, attitudes towards information, languages, perception of time, and career paths differ. The imperatives that drive scientists and policy makers are also different, along with their production processes and what they consider to be good evidence.⁶

This paper addresses a fundamental question in evidence based policy making, namely, whether scientists and policy makers can work together. It takes a parody approach to the question. The paper stereotypes certain stakeholders in ways they may not appreciate. This is simply a rhetorical device. It is not our intent to ridicule anyone or any group. We seek to highlight issues and to suggest solutions that are of practical relevance to the reader, using language that is hyperbolic. The paper is intended to provide useful guidance in navigating the gap between data production and data use.

Many of our suggestions are made in the form of questions, to stimulate further debate in this very fundamental area of evidence based policy making. We hope that as a result of this debate, new opportunities for partnerships between scientists and policy makers will develop, further advancing evidence based public health policy and practice.

THE SCENARIO

The goal of scientists is to advance science. A key activity is to put out papers (that is, to add to the body of knowledge). They are less interested in broad issues, for example, the “big picture” social or policy aspects of their work. The target in the research world is “publications, patents, and professorships”.⁷ In essence, scientists “publish or perish”. To publish, scientists seek to dig a research hole so deep and narrow that “outsiders” (scientists and non-scientists) would have neither the data nor the expertise to compete. Infinitesimally, then, a scientific expert is someone who knows more and more about less and less, until finally knowing (almost) everything about (almost) nothing. Their specialty is pointing out flaws in studies, especially those done by others. Their ideal is to become a “Dr/Prof Expert”. They search for truth, by using a rational model.

The goal of policy makers is to obtain popular support. A key activity is to put out fires (that is, to manage political crises—while many would seek to be more proactive than this, solving crises is what they may end up doing most of the time). They are more interested in broad issues, for example, solutions that can be generally applied to a wide variety of problems. Political correctness is a key driver. The target in the policy world is “policy, practice, and people”.⁷ Because of the complexities of policy making, and the amount of meetings and briefings, they have very little time to consider original scientific publications. Their specialty is reading “bullet points” (why would they want to read much of the esoteric material generated by possibly well meaning scientists?). Their ideal is to become a “Mr/Ms Fix it”, especially across numerous portfolios. They search for compromise, by using an intuitive model.

Scientists speak their own language that normally consists of at least some Greek letters and mathematical symbols. Their language often requires “translation” before it can be understood by non-scientists, or even scientists in a different field. Scientists often stamp a standard clause at the end of their research publications, “in summary, our research indicates that more research is needed”. This is, of course, both exculpatory (“don’t blame me if this isn’t correct”) and self serving (“but if you give me more money I might be able to give you a better answer”).

Policy makers speak their own language that normally consists of acronyms, which in turn are defined by other acronyms. Communications are often for a closed audience and driven by unpublicised political agendas. They often include multiple signatures (or no signatures) at the end of the reports they prepare, and then stamp “confidential”. While policy makers do conduct research, it is rare to see their findings published in scientific journals, if released at all.

Time works wonders for scientists. In general, it is believed that the longer it takes to do a research study, the better the research quality is. Scientists have a joke, “the way to live forever is to find a research question so important that one cannot die without finding the answer, then never work on it”. Scientists usually spend their entire research career in one narrow subject area, to build up their expertise and track record, as well as national and international reputation in that area.

Policy makers work to a different time scale—time is everything. Answers are always needed instantly and the timetable often has precedence over the quality, as they must have prompt and firm opinion to look credible. This is reflected in the classic policy makers’ joke, “I have made up my mind, don’t confuse me with the facts.” Policy makers usually have short tenure managing projects, and will move on quickly to other files, to build up their repertoire of expertise in a wide variety of different areas.

THE ISSUES

The incompatibilities between scientists and policy makers are very real ones. If scientists and policy makers are to work together, they must know each other’s strengths and weaknesses, as well as likes and dislikes. There are a number of key issues that must therefore be addressed.

Scientists and policy makers often lack trust and respect for the respective parts that they play. Scientists have a lack of respect for those who are not scientists—science is an exclusive club, and scientists live in a culture that reveres scientific ideals. They often consider their research to be cutting edge, to be reviewed only by their peers, and find it difficult to conduct “directed” research. They resent the power of policy makers to control research funding and the frequent misuse that is made of scientific data to fulfil a political policy agenda. On the other hand, policy makers resent the arrogance of scientists, the seeming self fulfilment of much of their research, and their tunnel vision approach to the world. They often view scientific input as untimely, less than relevant, and impossible to understand or contextualise. This is well described in the “two-communities thesis”, which postulates the existence of two camps (scientists and policy makers) that lack the ability to take into account the realities or perspectives of one another.⁵

Additionally, scientists and policy makers have different views of what constitutes evidence.⁸ Many scientific results are quantitative and can be assessed in rigorous, repeatable ways. Scientists obsess about research methodology and the “levels of evidence” gathered through different study designs, such as clinical trials and observational studies. Policy makers are often more informal in their assessment of information, even that of a quantitative nature. They look for important information based on quick reflections of reality for policy making, for example, poll results, opinion surveys, focus groups in marginal electorates, anecdotes, and real life stories. They operate on a different hierarchy of evidence—their “levels of evidence” may range from “any information that establishes a fact or gives reason for believing in something” to “available body of facts or information indicating a belief or proposition is true or valid”.

With respect to the production and use of evidence, an issue is whether research is a “retail store”—that is, whether scientists can and should cater to the needs of policy makers.⁹ Policy makers are frustrated because scientists cannot give them a quick, clear, simple answer. Scientists are frustrated because required data may not exist, or they do not know the answer or want to admit problems with their studies, or they cannot explain their complex findings in a simple language. Policy makers believe that much of the research being conducted is pointless, which is probably right as the motivation on the part of the scientists is often scientific curiosity and the desire to publish. The nub of the issue in using evidence differently lies in the differences in decision making imperatives. Not only might scientific evidence conflict with values and beliefs of policy makers, but the policy maker uses evidence in the battle to control problem definition and policy solutions.¹⁰ Policy makers thus look for evidence to support their claims, and thus systematic bias occurs in the way that policy makers look for and use data. Another facet of this issue is that policy makers are often concerned that highlighting knowledge gaps will reduce support for their programmes. They thus end up making uninformed decisions. In summary, the two camps differ in their attitudes towards “use of evidence”.⁵

Scientists have long attention spans; policy makers cannot afford this. The whole process of science is a very slow revving and cumulative one—science builds on previous research findings. Many scientific studies need careful and long term advance planning and preparation. It can take years, depending on the questions, to generate and assess the science on an issue and if one does not have the skill sets on hand, add many more years to the timeframe for training and development.

Further complicating the issues are the weaknesses in logic in both scientific and policy making approaches to setting priorities and achieving outcomes. Science and policy making are chaotic in different ways. Most scientific research is derivative, and unhelpful from a policy perspective. The 23rd paper on smoking and a certain disease may still be published, but it is not really advancing science unless the study is somehow considerably better than previous studies; too often, it is not, a phenomenon known as “circular epidemiology”.^11,12 In other words, there is a lot of indifferent or “junk” science out there, and policy makers are clever enough to recognise this. Policy making is built on a history of related policies, but is also reactive to numerous and competing stakeholder demands. At the end of the day, policies are the result of compromises and are constantly framed and re-framed in response to changing contexts.¹³

A further issue is that of the need for scientists to hedge their findings—scientists recognise the limitations of their data and are striving to provide proof “beyond reasonable doubt”—but policy makers need a simple one line answer to what are often, at least to the scientists, complex issues. In presenting their results, scientists traditionally rely on so many caveats that policy makers do not know what to believe. Policy makers frequently have to exercise moral judgements in the face of uncertainly, so decisions are taken “on the balance of probabilities”. They usually have plenty on their plates, and gravitate towards evidence that speaks to their own experiences, or that of their constituents. They seek a “one size fits all” or “cookie cutter” approach.¹⁴ Policy makers want a “bottom line”, but scientists are uncomfortable giving one.

The relative accountabilities of scientists and policy makers are also worth noting. Scientists are essentially accountable to editors of peer reviewed journals and grant funders. They may be interested in policy but, at the end of the day, are not required to focus on issues that have policy relevance or application. On the other hand, policy makers are usually accountable to political parties, government, and taxpayers, if not the voters, and must focus on things that are consistent with political agendas. Complicating this however is the increasing pressure on scientists to comply with views of governments that are increasingly responsible for setting priorities in the way research funds are allocated. So smart scientists will have their research proposals reviewed by policy makers before submitting their grant proposals.

There is unfortunately no correlation between the quality of science and the policy derived from it. Good science does not always guarantee good policy; bad or even no science does not necessarily lead to bad policy. It is true good policy does not always depend on waiting for good evidence. For example, condom promotion makes good common sense when setting policies to tackle sexually transmitted diseases. On the other hand, having a policy, no matter how carefully thought out, is no guarantee that it works. Having a policy for clean water, for example, does not necessarily make the water clean. It must be realised that science is needed both to help develop the policy and to evaluate the policy. Science is before and after; policy is the meat in the scientific sandwich.

A further issue is in the public image. Scientists are often respected as “wise people” and free from political and economic interests. Policy makers are often regarded as “powerful people”, but not necessarily respected. The list goes on and the incompatibilities seem to be growing. It is not that either scientists or policy makers are “wrong” or “bad”. One responds to scientific rationality, while the other responds to political rationality.⁶ Furthermore, the societies within which they work also have norms and expectations, which might be considered “cultural rationality”. It is when these competing rationalities come together then the seeming incompatibilities are resolved and evidence based health policies are adopted.

THE SOLUTIONS

There are success stories of scientists and policy makers who work together, such as the European Observatory on Health Systems and Policies,¹⁶ and the work of scientists with policy makers at the Milbank Memorial Fund,¹⁷ both of which use the “knowledge brokerage” mode.^18,19 In the UK, the network of public health observatories is already bridging the gap between policy and academic communities.^20,21 Knowledge brokerage also simplifies the information—a good example is the Health Evidence Network set up recently by the World Health Organisation, which goes a step beyond Cochrane style systematic reviews²² and tries to come up with one page policy briefs in response to questions posed by policy makers.²³ There are also successful examples of “doing the science within government”. Centres of scientific excellence do exist within government departments, where scientists and policy makers jointly set the research agenda.^24–28 And many research units in many countries receive core funding from governments that necessitates a dialogue between policy makers and academics. Many researchers now send research proposals to policy reviewers in addition to the more traditional academic peer review route in recognition of the valuable input that these reviewers can provide.

In other cases, things turn sour. Many policy makers adopt evidence that supports what they have already decided and are less keen on evidence when it conflicts with this. For example, we learnt about a university research unit funded by the government that was evaluating a programme, but the government had already started rolling out this programme long before even the participants had been recruited into the randomised controlled trial. Some researchers found a gloomy picture on the research into practice front. For example, vignettes contain little sign of decision makers’ commitment to scientific evidence.²⁹ Innvaer et al reviewed 24 interview studies with health policy makers (a total of 2041 interviews) concerning their perceptions of the use of research evidence in health policy decisions.⁵ The most commonly reported facilitators were personal contact (13 of 24 interview studies), timely relevance (13 of 24), and the inclusion of summaries with policy recommendations (11 of 24) (table 1). The most commonly reported barriers were absence of personal contact (11 of 24), lack of timely relevance of research (9 of 24), and mutual mistrust (8 of 24). The question then is how to fully recognise the incompatibility problems and to promote successful experiences in the collaboration of scientists and policy makers—that is, promote facilitators and suppress barriers. Also, the solutions to the questions run deeper than simply putting the scientists and policy makers in personal contact, or just asking scientists to provide timely and relevant findings.

We raise the following questions concerning possible solutions.

Should there be incentives for scientists or policy makers if they take the initiative to build a dialogue with their counterparts? Research funding does not provide for information dissemination. Engagement with the public, including policy makers, is not rewarded. Incentives are also needed to encourage policy makers to acquire a higher level of scientific training than is the present norm. Scientific thinking and results can be dumbed down only so much before becoming meaningless. On the other hand, at least some scientists need to develop a sense of the “big picture” and work on ways to make scientific work understandable and usable by intelligent lay people. Unfortunately, none of these will happen unless there are incentives for them. The current reward mechanisms simply do not work optimally to encourage policy makers and scientists to work together.^30,31 The need for a partnership was pointed out in the 1998 Johns Hopkins Symposium on the “Translation of Epidemiologic Evidence into Public Health Policy”.^32,33 However, what incentives are there for this partnership? New incentives may have to be created for changes to take place.

Should there be knowledge brokers (or translational scientists) to go between scientists and policy makers?^18,19,32 Scientists and policy makers are two highly specialised groups. Perhaps they should be left alone to do their own jobs. Knowledge brokers may serve as catalyst to look for, and nurture if possible, the relationship between the two groups. Linkage and exchange can occur along mutually beneficial lines paying specific attention to opportunities for specialisation that maximise the benefit of both groups. In other words, they can ensure that policy makers are using “the right science”, and that scientists are doing “the science right”.³⁴ For example, integrating and synthesising scientific information into knowledge, good knowledge brokers may be able to say to the policy makers who are swamped with information, “Here is the list of the top 10 major issues in this country according to current knowledge”. The knowledge broker may then turn to the scientists, “Give me the science on what works to tackle these issues” and then produce an inventory of evidence based best practices. The demand for evidence and information should, ideally, come from the policy makers themselves but this often does not happen. A critical role of the knowledge broker is to “translate” this demand and “re-translate” information that comes from the research community—in a way that is understandable and transparent, including evidence that is “in conflict” with what policy makers have already decided. Knowledge brokers can also assist scientists to think about “lighthouse” indicators when attempting to attract the attention of policy makers. These indicators are the guiding lights for policy. They assist government in navigating options and choices for strategic planning and policy. They are highly visible and stand out from the maze of distracting data. They provide information that is highly accurate and reliable, allowing quality decisions to be based on quality information. Quality lighthouse indicators will focus the attention of the captain of the policy ship in the face of a proliferation of distractions that can otherwise lead to unacceptable, even terminal, outcomes for policies and programmes.

Should there be organisational capacity building interventions? These may include mechanisms and processes within organisations to ensure there is input from researchers and policy makers.³⁵ For example, the American Association for the Advancement of Science (AAAS) has a programme where scientists are actively encouraged to enter the policy making arena.³⁰ There is a range of possible workforce development approaches and appointment strategies—for example, requiring diverse skills, secondments, job rotations, dual appointments, liaison units, etc. Skills required of policy makers in the future will probably be different because the world of public administration is changing.³⁶ How about a chief knowledge officer?³⁷ Each organisation needs to manage knowledge as well as it manages its other resources—think of a chief administrative officer or a chief executive officer. Why should a policy maker talk to a scientist who may not even understand the policy question? Just give the chief knowledge officer a call.

Should research be defined far more broadly than the biomedical community conventionally believes?³⁸ In a broader definition, it can be argued that research is an investment, not a cost; that all countries should have a health research system that drives health sector reform; that research should be applied to improve health equity; that research must be conducted according to universal ethical standards; and that the results of research should be accessible to all.³⁸ A key challenge in public health is in strengthening health systems.³⁹ To strengthen health systems, there is a need for human resource development through, among other things, strengthening capacity for operational research in health systems development. Partnerships are urgently needed between government policy bodies and academic/research organisations experienced in this area.³⁹

Should the knowledge transfer starting point be re-defined? Do most of the current “research to policy” efforts focus on the wrong “starting point” (that is, the researchers)? Perhaps the civil society has a vital—and so far neglected—part to play in setting research priorities.³⁸ There is an important role of the citizen or community in evidence based policy, for example, in the increasing community engagement and citizen participation in health systems and the increasing trend towards including patient and public input into research. It has been pointed out that about 90% of real world problem solving is spent: solving the wrong problem; stating the question so that it cannot be answered; stating questions too generically; trying to get agreement on the answer before there is agreement on the question; solving a solution.⁴⁰ Research funders and policy makers have to become a lot more skilled at ensuring that scientists spend a lot of their time researching the questions that have the greatest potential to improve the society. They should be encouraged to fund synthesis research and impact assessments in support of policy decisions. Research can improve the robustness of decision making. The trick here is to connect science with policy, and policy with science. It is desirable to have both “evidence based policy” and “policy based evidence”.⁴¹ In other words, policies should be based on evidence, and once policies have been formulated, there should be evidence on how to achieve the set goals, and to develop, implement, and evaluate needed strategies. There is no better way than to have policy makers intimately engaged in the science.³⁰ However, one must be careful to make sure that “evidence based policy making” does not become “policy based evidence making”—that is, creating and selecting evidence that suits and justifies certain formulated policies. Sometimes policy makers want to stretch the interpretation of research findings to reinforce the “validity” of the policies they are already decided upon. There are potential problems when scientists get too close to policy, for example, concerns about loss of objectivity and freedom to criticise government policy, and how to guard against this. Research dollars often come from governments. The question is whether and how the scientists can resist the ideological pressures from their life blood funders.

Should the accountability to peer reviewed journal editors (or some form of scientific peer reviewers) extend to policy makers in the government sector? Increasingly policy makers are respecting this benchmark for quality of service, in part because of the fact that scientific literature helps to widen the sphere of influence of policy. Successful and unsuccessful experiences in policy making should be published in scientific journals, for critical peer review and for reference by other scientists and policy makers. This dual accountability can provide a powerful opportunity for scientists to have a unique impact on the way government ultimately does business—and on the way that public health programmes are implemented. The society will gain by making both researchers and policy makers more accountable.

Should it be recognised that scientists and policy makers are not equal partners, and therefore additional work must be done to promote their dialogue? From the scientists’ point of view, the scientist/policy maker relationship is a one to one—that is, application of research results in policy actions. But from the policy makers’ point of view, the situation is a many-to-one relationship. They must respond to the wants of multiple stakeholder groups, such as the private sector and the general public. Scientists are only one of many groups of people that they deal with and listen to. In that sense, one could say that scientists and policy makers are not equal partners. Policy making is making judgement in light of uncertainty, and not always done on a scientific basis. A challenge for scientists, if they are interested in influencing policy (which is not always the case), is not only to convince the policy makers, but also to convince and mobilise people. This calls for understanding the different backgrounds for policy decisions, and developing many kinds of communication and networking skills to address counter argument and communicate messages well. But even more important is to influence public opinions, intentions, and behaviours, through good media communications and partnerships with different stakeholders. Knowledge brokers can assist scientists in this regard.

Should it be acknowledged that it is too simplistic to think that policy making could ever be purely or perhaps even largely based on scientific evidence? Policies are decisions on what policy makers want to achieve. In addition to scientific evidence, policies are also based on values, emotions, and the wishes of interest groups, for example. The reality of how decisions are made dictates that scientific evidence is only one consideration among several. Such evidence can even in its best be only background. In some cases, it is perfectly possible for wise policy makers to develop good policies without research.⁴² In other cases, policy makers listen more to the voters than to the scientists. We should perhaps admit this and not set unrealistic expectations for the role of scientific evidence, and acknowledge that, on the other hand, failing to grab accessible evidence may delay intervention opportunities. For example, it took 263 years after the discovery of the preventive value of citrus juice against scurvy before sailors’ shipboard diets were routinely supplemented with it at the end of the 19th century.⁹ The link of smoking to lung cancer was found in 1950 but it was not until 1957 that any legislative action was started.⁹ How long will it take to tackle the current epidemic of obesity if our will to intervene awaits the delivery of perfect evidence that proposed solutions will work? Thus, the balance between action and further research is an interesting and important one. When do we need policy decisions and when do we need more research? It is even possible that scientists are somewhat biased in their advice because they want more research money. Research is too often used as means to put away the needed policy decisions.

FINAL WORDS

It is our hope that scientists and policy makers can draw lessons from ecology: a science that studies the co-evolution of different populations in their environment.⁴³ Ecologists will tell us that populations can evolve together antagonistically or complementarily: in both cases the populations adapt both to survive and to work effectively in an environment shared with the other population. The term “mutualistic relationship” is used to describe the co-evolution of two populations in which both benefit.