Key Issues surrounding EBP

What evidence is used in the policymaking process?
What is clear from the literature is that evidence is an ambiguous term. We take the view that evidence-based policy should be based on systematic evidence; that is, research-based evidence. The key to this is that we adopt a very general, though widely accepted, definition of research as 'any systematic effort to increase the stock of knowledge' (OECD, 1981). Thus we include all kinds of evidence, provided they have been collected through a systematic process. This may include critical investigation and evaluation, theory building, data collection, analysis, and codification related to development policy and practice. It also includes action research, i.e. self-reflection by practitioners orientated towards the enhancement of direct practice. In effect, we are talking about research-based evidence-based policy - for ease, we stick to the term evidence-based policy.

It would be a mistake to assume that in reality all forms of evidence share equal importance, relevance or weighting. Departments and units within the government tend to make hierarchical judgements in choosing what evidence to use, where and how - these decisions are often deeply embedded in assumptions regarding validity and power. Often, it is only hard evidence (or empirical data) that is used. This is often narrow-minded: tacit forms of knowledge, practice-based wisdom and, perhaps most importantly, the voices of ordinary citizens - the 'voices of poor' - are often equally useful. The implication is therefore that an EBP approach should take into consideration a wide breadth of sources of research, not just hard evidence.

What issues should governments consider when trying to identify what evidence is useful? Recent work (Court, Hovland, and Young, 2005; Shaxson, 2005) suggests that governments should consider:

• Accuracy: Is the evidence correctly describing what it purports to do?
  
• Objectivity: The quality of the approach taken to generate evidence and the objectiveness of the source, as well as the extent of contestation regarding evidence.
  
• Credibility: This relates to the reliability of the evidence and therefore whether we can depend on it for monitoring, evaluation or impact assessments.
  
• Generalisability: Is there extensive information or are there just selective cases or pilots?
  
• Relevance: Whether evidence is timely, topical and has policy implications.
  
• Availability: The existence of (good) evidence.
  
• Rootedness: Is evidence grounded in reality?
  
• Practicalities: Whether policymakers have access to the evidence in a useful form and whether the policy implications of the research are feasible and affordable.

How evidence is incorporated into policymaking
Policy processes ideally involve different stages: agenda setting; formulation; implementation; and evaluation. Evidence has the potential to influence the policymaking process at each of these stages. Different types of evidence are often needed for different parts of the policy process, and time considerations are likely to influence the mechanisms available to mobilise evidence.

For each different part of the policy process, we revise the work of Pollard and Court (2005) to outline some specific issues regarding use of evidence (see Table 1).

Evidence is not the only factor which influences policymaking
It is important to acknowledge that at each stage of the policy cycle a number of different factors will also affect policy. This occurs both at an individual level, e.g. in terms of a policymaker's own experience, expertise and judgement, and at an institutional level, e.g. in terms of institutional incentives, interests and capacity. There are also a number of constraints that will limit the extent to which evidence can affect policy, e.g. the pressure to process information quickly. Policymaking is neither objective nor neutral: it is an inherently political process. Nutley (2003: 12) highlights the fact that the interaction between policymakers and researchers is limited by the divergence of these two worlds. They use different languages and have different priorities, agendas, timescales and reward systems. As a consequence, a communication gap often exists.

There are clearly challenges here, but the consensus among researchers, policymakers and practitioners is that more evidence-based approaches to policy and practice are a positive development.

Back to Progressive Policymakers Toolkit index