ODI Logo
  ODI Home Page  
RAPID  Home
 
R0106 - TRISP Literature Review
Is your evidence robust enough? Questions for policy makers and practitioners

This article examines the reasons evidence for policy is needed. 'Evidence is a necessary, but not a sufficient, condition for any decision-making process'. It discusses where evidence is needed in the policy-making process, and the nature of the evidence base for strategy and policy. Working relationships between policy makers and their advisers are key: as policy makers come from a variety of backgrounds, developing a common language helps set discussions about the robustness of the evidence base on a sound footing. The article identifies five components of evidence robustness (from a policy-making lens); credibility, generalisability, reliability, objectivity and rootedness or authenticity. These characteristics are disseminated and applied to the processes of policy making.

Credibility: This relates to the processes of analysing and synthesising information in quantitative literature. Credible evidence relies on a strong and clear line of argument, tried and tested analytical methods, analytical rigour throughout the processes of data collection and analysis, and on clear presentation of the conclusions.

Generalisability (or transferability): This refers to the way in which we make inferences and therefore the ease with which it would be possible to take the evidence which has been collected for a specific purpose and use it in a different context, or to answer a different question. In some cases this will refer primarily to sampling techniques, however in others it will refer to the broader framing of the issue and the policy question. It is particularly applicable when pilot studies precede a wider roll-out

Reliability: This relates to whether or not we can depend on the evidence for monitoring, evaluation or impact assessments - planning for the lessons learned approach.

Objectivity: Questioning the bias in the evidence base to deepen or understanding of how it conditions our interpretation of the evidence for policy.

Rootedness (or authenticity): This implies more than context, process, bias and the quality of information. Rather, it is about understanding the nuance of the evidence, exploring assumptions with an open mind, encouraging others to question the status quo, and thinking about who uses what evidence for what purpose.

(based on article's abstract)

 

Author: Shaxson, L.
Date: 2005
Type of publication: Journal article
Publisher: Policy Press - Evidence and Policy: A Journal of Research, Debate and Practice, Vol. 1, No. 1, pp.101-111.
Document:
Available online at: http://www.ingentaconnect.com/content/tpp/ep/2005/00000001/00000001/art00006

Back to EBP bibliography index

 
Last Updated: 13 January, 2009
www.odi.org.uk