A practitioner´s understanding of the key factors in play to promote evidence based policy processes

I am going to share with you a few things that I have learned about why we use evidence, who is involved in the policy process, when evidence can best be used in the policy process, what it is used for and how it is put to use. I will then talk about the main challenges faced by actors who are trying to use evidence, the approaches that are being used and some conclusions and recommendations. In addition to my own work on this, since I am not dealing on a daily basis with scientists (I deal more with social scientists), before coming to give this presentation I spent some time interviewing scientists on their experiences, or lack thereof, in engaging with policy processes and I have brought some of their perceptions to this.

Before we start, I want to set out the generic policy processes that I have in mind, as well as our experiences when we go out into the field, and to be explicit about the fact that there are different stages at which evidence can be brought to the table. Not all of them take up evidence as easily as others, but there are definitely significant opportunities for the actors in this game to get involved in (i) the agenda-setting process and in (ii) the building of alternatives. To a lesser extent, in my experience, there are also opportunities to get involved in (iii) the choice of possible solutions. And only sometimes (in a more limited way) there are opportunities to get involved in (iv) policy design, (v) policy implementation and, finally, in (vi) policy evaluation.

Why consider using evidence at all in policy processes? Besides the academic literature that one can find - and this is not an academic presentation - evidence can be used in a very practical sense to improve policy, which in turn improves outcomes. If we make fewer mistakes when we produce policy recommendations, we usually have better social results. Secondly, at the level of public debate, the quality of our discussions is improved when the opposition and the government have evidence to support the positions that they hold and we usually have a more informed public. Thirdly, evidence allows for more rigorous testing of certain social hypotheses, such as whether or not having schools open on weekends reduces child crime rates. When we use evidence, we give more value to the evaluation of ideas that are tested out in the field.

Who are the actors that I mostly get involved with when considering the policy process and the use of evidence? The first group are obviously political actors. Unlike the UK framework, in many developing countries the government and the legislature are separate organisations, both worth trying to influence. In my own experience I have tended to view the government and the opposition as the key actors to which we direct our efforts, which is probably valid for many of our southern hemisphere systems that are more presidential. Our systems are usually very executive in terms of decision-making, and although we have three branches of power, the president usually has a very significant weight in the policy process. To a certain extent, the opposition can play a role, particularly when it has gathered enough power to become a future alternative government. Then, of course, there are the bureaucrats and I have put them together with politicians because sometimes they share the same prejudices with regard to evidence, civil society organisations (CSOs), the private sector and actors from outside the policy process who want to come into the policy process. Then there are CSOs ranging from think-tanks to grassroots organisations. The media is more a consumer than a producer of evidence and it plays a key part in influencing how politicians and bureaucrats make decisions. Then we have university scientists and research centres that are also producers of knowledge, theory and evidence, but which sometimes do not have all the links that they might want or expect with the political system. International organisations and donor agencies have a very varied influence across the southern hemisphere. In some countries, they have a huge and very significant weight and in other countries their influence is much less. Then there is the private sector, also with significant differences from country to country in terms of its capacity to influence policy making. Of course, there are many more actors but, at least from a practitioner's perspective, these are the ones with whom I usually need to engage.

When do these different actors best use evidence? In my experience in Argentina, it is clear that the media, CSOs and the private sector have had more opportunities to influence the agenda-setting stage than some other aspects of the policy process. So, in terms of the stages that we mentioned before, agenda-setting is one of the primary stages when evidence is brought to the table. In the policy design and formulation stages, decisions in our systems are usually reserved to politicians and thus depend less on what we can and cannot do from outside the political environment. These moments are usually influenced more by political outcomes and by whether or not something is convenient for elections, or for coalition building than by evidence. Instead, evidence can play a role in policy implementation, particularly evidence that comes from practical knowledge of what works and what does not work. And to some degree, part of the evaluation and monitoring process also relies on capturing and using evidence, but in our cultures in Latin America, at least, we tend not to do very systematic evaluations of policy choices and policy impacts.

That was a little bit of the picture of the different actors, and the different instances in which they get involved in the policy process. It is evidently a complex situation with a lot of interaction among different players who have varying weight at the different stages of the policy process.

What is evidence used for? Why do these different actors make the effort of gathering knowledge or information and bringing it to the table? In my conversations and experience, I found a number of different reasons. One is that if you have evidence and information about a given issue, you get easier access to the negotiating table. Sometimes politicians need the information you have and that gives you entry to the policy-making process. In our case, the think-tank I work in has lots of information and evidence on the results of educational financing in the provinces. For example, we have proof that the poorest districts get less money and we have information about what that money goes to. Because we are the ones who have the most information, the government needs to talk to us and call us to the table, so that when it is making a decision about how to transfer resources to the districts, it has this evidence or information to act upon.

On the one hand, then, evidence is used to gain access to the policy process. Secondly, as my colleague from Oxfam mentioned before, evidence is often used to lend legitimacy to decisions that the government wants to take, sometimes with more honesty in the effort and sometimes with less. By this, I mean that sometimes the government wants to know what the evidence is in order to make a decision and at other times it has made a decision and just wants the support for that. Nonetheless, there is definitely a rationale that goes into legitimising policy decisions.

Thirdly, it is also used by organisations, the private sector, CSOs and universities, to gain credibility and reputation. If I have the insights, the knowledge and the social information or proof that things do or do not work in a particular way, I can more easily gain space in the media and in public debates. Usually, if I have done this for a significant and consistent amount of time, I become a source of reference. When many young civil society organisations and think-thanks start playing this game, they try to produce good documents and pieces of work and then make a range of efforts to get someone to pay attention to that. They are basically screaming and shouting and if they are lucky, someone will take their piece of work and give it some consideration. After they have done that for several months or even years, what they are doing becomes more visible and the relationships are already built, so that people who need that information, whether they are decision-makers or journalists, call and ask if they have anything on child labour or human trafficking or whatever.

By this point, these organizations have probably already become a source of consultation. As they sustain that through time and build up their credibility and reputation, you eventually come to the stage that I think most people who want to influence the policy process want to reach, which is that when you come up with an issue, and politicians pay attention to it. At least in my organisation, this is a long way from what we can do today, but it is a game which is fascinating to watch when you have organisations such as Oxfam that decide to come out with an issue and the politicians respond, so that the organisation really sets the framework for discussion and policy choices.

Another reason for using evidence is that it allows the different parts of the policy process to become more integrated. I have seen many times in our own work that you have social scientists and people with PhDs in healthcare or education analysing a public problem and designing a solution, but they are not the ones who can take that into the policy process. You usually need the lobbyist or the policy activist to go out onto the street or to go the Congressman or woman's offices. Then, once that policy has been adopted, you need the consultants to go in and work with the budget, change the rules of the administration and engage in the delivery of the service. With evidence it is possible to integrate these much more. You can tell the PhD researchers who are at an early stage in the process what might or might not work according to budgetary or political constraints. You can inform the activists about the sources of legitimacy for the things that they are promoting. In this sense, evidence allows for integrating the various parts of the policy process and it often serves to increase effectiveness, by bringing untapped resources to the table or by reducing the amount of mistakes made by the policy-maker. If we tell the policy-maker that we have evidence that providing micro-credit through certain types of institutions in the local community has been tried before and did not work, we spare them from making the same mistake twice. That kind of evidence usually draws on local knowledge. Sometimes the politicians designing policies 5000 kilometres away do not know the constraints of the local actors and do not know who is who and how they operate. Bringing evidence to the table allows for better policy decisions in that sense.

Another use of evidence is in translating complex policy issues into people's every day concerns and, directly linked to that, communicating in an accessible and meaningful way. When someone comes up with the fact that every cow gets two dollars a day, that suddenly makes a complex subject such as subsidies into something that everyone can understand and feel concerned about. So these are some of the reasons why researchers, civil society organisations and private sector organisations make the effort to gather evidence and try to take it to the table.

How do they do it? Once they have evidence in their hands about a situation or policy outcome, how do they get that evidence into the policy-making process? There are some very practical things that organisations and people usually do, including building coalitions and networks through which they can promote their policy positions. They also provide training that uses their evidence in the lessons, classes or workshops that they give. They suggest policy alternatives and come up with documents that say, for example, if you have to decide how to expand pre-school childcare, you have choice A, choice B and choice C, and they take that to someone in a government office or the legislature. Alternatively, when a document comes from the other side, they comment on the proposals. Last week, for example, one of our key legislators in Argentina was proposing legislation on human trafficking and so they asked us if we had any comments. Since we are not experts, we passed it on to those experts who would know better, and then get their comments back to the legislator showing how analysis and evidence can be put to use by suggesting or commenting on draft legislation.

Most think-tanks, organisations and industrial associations organise public seminars at which different points of view and evidence are put on the table. There is a lot of work being done to put evidence out there, through the media, through the web and through publications. There are lobbying efforts that are literally individual face-to-face meetings with key decision-makers to convince them about the strength or weakness of a position. Evidence may be brought to the table by working together with the government on particular projects, such as reform of military education, where we ask what has been proven to work in other countries and we bring that documentation to the table. By working together on a project, we usually manage to put that evidence up for discussion. Finally, those organisations involved in direct service provision (such as those providing healthcare or education) also bring first-hand experience and practical knowledge to the table and this is something that often influences the design of policy.

What are the main challenges faced by those who want to use evidence to influence the policy process? On the one hand, there is often a lack of staff capacity or a lack of capacity to produce adequate analysis, for example, when organisations do not have scientists or researchers who can understand certain complex problems, or they do not have the ability or time to process the information that others make available. More commonly, they do not have the ability to communicate their findings and they produce fifty page reports that no decision-maker has time to read. In other cases, they lack time or funds and they are usually running behind projects that do have funds, so that they do not have any time to dedicate to other relevant public policy problems. Sometimes they lack the knowledge or the tools, or the knowledge is simply not available in the country. In Argentina, for example, if you want to try to produce statistics on changing crime patterns during the past fifty years, there are no statistics available and it is impossible to find either public or private information on the subject. There is a lack of public knowledge and information about these things.

In many cases, the policy process is totally inaccessible. For example, in terms of distance: an organisation that is 3000 kilometres from the city of Buenos Aires, where most national decisions are made, is likely to find that it cannot influence policy because it cannot bring its evidence to the decision-making table. Sometimes the process is inaccessible because of language barriers. Sometimes it is inaccessible because the policy-makers are closed to new information and are not interested in whatever other parts of the community have to say. Other times, when there are funds, knowledge and an open policy process, the problem is a lack of credibility. One of the leading organisations in Argentina was doing public advocacy and campaigning work and had a very good proposal about changing part of the special pensions for public employees. However, at one point a scandal came out that the president of that NGO was apparently also receiving a special pension, so they lost all credibility in the debate and no-one would consider their proposal after that. In other cases, it may be that the proposal or the evidence that you bring to the table is not useful to the decision-maker, because it is not timely, not written in a way he understands, or not easily applicable to the problem he has to deal with, so you are out of the game.

Corruption is another big limitation. You might have the best evidence and the best intentions, but if there are other interests on the table and decision-makers are basing their decisions on other incentives (usually related to private sector benefits), there may not be much you can do to bring that evidence to use.

In the face of those key challenges, what are the different approaches that are being used? As we heard from Oxfam, campaigning is a very typical strategy. Much of the work at ODI and at other organisations has documented these different approaches. The 'boomerang approach' (I thought this was a smart choice of words in one of ODI's pieces of work) is when you throw a problem outside your district or outside your country to an international organisation or arena, or to regional organisation, and you have it come back from a different player. So, for example, we have had a chance to sit down with the World Bank and the InterAmerican Development Bank and convince them of a few issues so that when they then sit down with our government, it is the Bank that is advocating for what we are concerned about. In this way we used a strategic opportunity to put an issue out to a foreign "partner". If I wanted to do anything about the situation of prisons in Argentina, for example, it would be much effective for Amnesty International to complain about them than it would be for me to do so.

How do I convince a foreign actor to take on the cause that I have? Obviously by showing them that they stand to benefit with arguments and examples. Pilot projects can be useful for this. We had a very interesting pilot project, for example, to provide free legal services in the country. We expect that the lessons we learned from that will be useful for decision-makers in other places when they decide to expand access to justice for many of the under-served population.

Another approach is to try to increase the capacity of organisations to map and understand politics and policy. After six years of working, this was the first year in which part of our team decided to take a proper look at who the political actors are with whom we have a relationship, how that relationship is built and sustained, where we are lacking those relationships, whether or not we have access to the key decision-makers we need to influence, and why. Developing that capacity to map and understand politics and policy-making is a very useful asset for organisations. However, it is not enough to know what you have and what is out there; you also need to build trust and relationships with decision-makers, and this is something that researchers and scientists usually have less of an interest in, or less of an ability to do. It is not the easiest thing in the world to sit down with a politician and to have to go through two hours of conversation about things that are totally unrelated to your objectives for the meeting, but it is part of the way things are done and there are rituals to the way that many of our politicians operate. Doing this is part of building trust with them - at least in several Latin American countries. A lot of politics is done after six or seven in the afternoon and late into the evening and if you are not at the table with them at that dinner, you will not manage to get them to hear your point of view. You have to build this trust and these relationships, preferably in advance so that you already have that relationship before they are in office or making a critical decision. It is much harder to try to influence the president once he is president. Five years ago, when he was the governor of an unknown province or the head of an unknown agency, it would have been so much easier. So it is important to try to develop those contacts, relationships, skills and abilities.

It is also vital to increase the capacity to gather evidence that is credible and relevant. For policy-makers, it must be possible to generalise on the basis of that evidence. Evidence is not very useful to them if it is only applicable to an indigenous population in a district of 500 people that is 5000 kilometres away; they need information that can be replicated in other districts.

As we heard from the other presentations, there is a need to communicate effectively, whether that is in private with the prime minister or president of Pakistan, or to a mass audience about Making Poverty History. This is not usually a well-developed skill on the part of researchers and scientists. We need to work more and better on that. Lastly, an approach that is becoming more and more useful for many of us is to try to learn from what we did wrong and build more capacity to overcome our initial limitations.

So, what do researchers need to do? First, they have to try to understand the political context. That includes understanding who the policy-makers are, whether there is a demand for the evidence that we have at hand and what the policy process is like. Then, with regard to the evidence, they need to understand what the current theory is, what the narratives are and the stories around that evidence, and how divergent or how coherent it is. They also need to understand the links between evidence and the policy context, who the key stakeholders are, what networks exist and who the connectors are. These are some of the key questions that researchers need to know the answers to.

In practical terms, once they know these things, researchers need to go out and get to personally know the policy-makers. They need to identify those who are in favour and those who are against their positions. They need to prepare for the policy opportunities. They might have a great idea, but if it is not on the agenda and they have no way of putting it on the agenda, they will just have to put it in a drawer and wait for the right moment to be able to get that idea out there. This means looking out for policy windows. Sometimes a crisis in the media, or an emergency or a social situation, produces the opportunity to get something onto the agenda.

What steps does one take to do this? You work with policy-makers and try to seek out and get involved with the relevant commissions and committees in congress. You try to use strategic opportunism, which means being prepared and knowing when policy-makers might need you and then being present at the right moment. The next thing to do is to establish your credibility and provide practical solutions that policy-makers might use. Always try to have this source of legitimacy for decisions that you want to support, present it in the form of clear options and use familiar narratives that they can understand. This is done by building a reputation, by doing research that is very action-oriented, by having pilot projects to which you can take the decision-maker to show them that something works and, obviously, by increasing your capacity for good communication. You have to get to know others, develop networks and learn to work in coalitions.

Finally, how do you participate, build and sustain coalitions? You can do this by trying out partnerships and sometimes by opening up and participating in spaces that are not directly related to your key concerns, but which are a way of creating a relationship and getting to understand and talk the same language as potential partners. Identify networkers, because they are people who are much better than you or I at getting the message to third parties; they are much better salespersons. It may be that the best person to go out and defend pension reform is not the academic who studied the issue, but an old pensioner or a young child. Think of the key people that you might use and, obviously, use informal contacts as well. Science and research is usually more documental and formal in many of its processes than policy because academic rigour requires that certain steps be followed. When you are working in the policy process, be more open to using informal arrangements.

In conclusion, research is essential but it is not enough. Other work needs to be done to ensure that it contributes to development. The clear lessons are that you need to understand the political context to maximise your chances of impact, you need to figure out what evidence is needed and have it or produce it, then get it to the decision maker and you need to collaborate with other actors.

There is much to be done, and we have a huge opportunity and responsibility to use evidence to influence policy. We should live up to that responsibility and make the best of the opportunities ahead.