Evidence-based policy: myth or reality?

I will give you a small glimpse of how we make policy internally in DFID, as well as how we have tried to provide some incentives and a bit of a stick to the researchers whom we fund to help their research feed into policy more effectively.

First of all, I will give a little bit of background for those of you who do not know DFID. I have worked in DFID for the last ten years or so and, over that time, there has been a real and significant change in DFID. It came about with the change of government in 1997 and I think it affected our whole approach towards policy, in that we moved away from having a mandate to look at development outcomes in areas such as agricultural productivity or health systems and moved much more towards thinking about people. We had a very clear leadership saying to us that our overall goal was about reducing poverty, and I think that that has systematically fed through the organisation and changed DFID from being an aid programming administration to being much more of an advocacy organisation internationally. I think that is something which is quite important.

We have had three White Papers since the government came to office, helping to set out a high-level strategic vision. We have also had an International Development Act (2002) that has given DFID the flexibility to use its funds differently. Our funds are no longer tied to the UK. We have had large budget increases and we have just agreed to increase aid to 0.7% GNI by 2013, which will actually make us bigger than the World Bank. This will give us more opportunity for advocacy.

We have more than 40 country offices, so we have a very decentralised approach to the way in which we work. We are also working quite a lot at the multilateral level, trying to advocate for reform of the UN system so that it becomes much more effective, as well as looking at the role of the World Bank and trying to get rid of aid conditionality etc. We still have advisory groups with technical staff in DFID, but we have set up our own policy division, which is a kind of internal think-tank. There are currently around 200 people in the policy division (though that may reduce a little bit in the near future) and we really think it is important that DFID can start to generate its own internal policy and own that process.

Staffing is very important to us. We have changed the nature of the skills of our staff over the past ten years or so. One of the key advantages we have in DFID is that we have a lot of people with technical qualifications, but I think that it is our ability to get those people to develop technical ideas and turn those into collaborative partnerships which is key and which then moves those ideas forward into action.

Another important change is the way in which we communicate about what we do. Hilary Benn is passionately keen to ensure that we are informing UK taxpayers about the effectiveness of our aid programme, because if it were not for them and things like the Make Poverty History campaign, we would not have the budget increases that we are having. We need to demonstrate to the UK taxpayers and to our partners that we are actually being effective in this area.

In terms of the DFID policy environment, something else that has changed in the last ten years is that we are now much more centrally part of Whitehall, which is the UK government system. Within the UK government system there has been a greater focus on science and evidence-based policy, partly stimulated by the government's Chief Scientific Advisor, Sir David King, but also by Gordon Brown and the Treasury's interest in making sure that, overall, government is adopting evidence in its decisions.

I want to talk a bit about a recent report, which I think came out last week, from the Science and Technology Committee in the UK Parliament, entitled 'Scientific Advice, Risk and Evidence-Based Policy-Making'. The report is really a critical look at how the UK government uses science and evidence in its policy decisions. (You can find it on the House of Commons website if you are interested in reading the full report). The main conclusion drawn was that in government, it is really important that each department has a Chief Scientific Advisor who acts as a champion and a challenge to government on its decision-making processes. It is also very important that that person has senior links into the wider scientific community. In DFID, for example, we have Sir Gordon Conway, who is a Fellow of the Royal Society and a Professor of International Development at Imperial College. It is important that government has that pressure and that challenge from the academic community.

In the civil service, promotions and staffing are not necessarily determined on the basis of someone's scientific knowledge. Thus, the recommendation was that we needed to look at how senior civil servants in the UK government could become much more intelligent customers of science, so that they know what the research process is, know what the research can produce and know about evidence-based policy and the importance of that.

Something else that came up, and which I think is quite important, is that whilst the nature of policy-making is very short-term, the nature of research is very long-term, meaning that there is a difficulty in the two engaging. What they recommended was that in order to make our policy processes more long-term, we needed to make sure that there was an element of horizon scanning as policy is developed, so that we are looking into the future even though, by its very nature, some of that policy must be short-term and politically driven.

Public communication is another key issue that came up in the report. I think that this concern is built on some of the crises that we had around BSE and some of the other problems that we have had in the past. Government needs to be much better at communicating to the public about how it develops its policy, as well as translating this more scientific evidence into the kind of 'killer facts' that were talked about earlier.

Interestingly, the report also said that not all policies need to be evidence-based. We should not be pushing for a position where absolutely every policy that we produce is evidence-based; it is also ok to have politically based policy as well.

We need to be better at engaging with learned societies, including organisations like the Royal Society, and professional bodies. Government has moved towards using a lot more consultants and the report suggests that we should be using learned societies as well, linking better into academic communities, and working better with the media.

So those were a few things at the Whitehall level. There are guidelines now, which all government departments have to use, about how we generate evidence-based policy. I do not think that we always do it in the most effective way.

Turning now to DFID, we have a new division, as I mentioned, called the Policy and Research Division, where we are trying to link together the central research department, which is where I am from, to the policy teams. At the moment our policy is evidence-based, but it is not necessarily based on long-term research, so we are trying to look at how we can create the linkages between the internal policies that DFID makes and the long-term research agenda. I think this is a real challenge. One way in which we are trying to do it is by making sure that our policy teams are involved in the governance of our research programmes, so that there is a link between them and some relationship-building going on.

When we asked our policy teams what the most important things about research were to them, they said that it was not really the publications or the outputs that they were most interested in, but having access to researchers. They want to build up relationships with researchers and be able to use those researchers and their knowledge to help to develop good policy. Obviously, they cannot talk to all researchers, but that was one key area where they thought we could be doing more.

Increasingly, DFID's policy is based on demand. One of the major mechanisms we use in our country offices is the Poverty Reduction Strategy Paper (PRSP), which is a process where donors and the World Bank come together with national governments to develop a poverty reduction strategy over five years. I think this is an interesting bit of the policy-making process concerned with looking at the users, and one that has not necessarily been covered today. How do we get the voices of citizens engaged in policy development? How do we make sure that we are not only using scientific evidence, but are also basing that policy on demand? I think that this is a really critical area for us, so we have been working to help the media develop capacity to engage in public debate around issues relating to Poverty Reduction Strategies, as well as trying to mobilise civil society so that it is much better able to represent its constituents' views in the policy process. At the same time, we have something that we call technical cooperation, where we commission short term studies. Particularly at the moment, we are commissioning quite a lot of work in support of PRSPs around drivers of change and what the political processes are in those areas and we are feeding in that technical input.

We also do quite a lot of advocacy work on reforming the international system. For example, three of four years ago I was working with FAO and at that time in DFID, we were trying to promote an approach called the Livelihoods Approach, through which we were trying to advocate that if you are working on agriculture, you need to think not so much about agricultural yields, but about people's livelihoods and the range of things which interplay around that, including agricultural productivity. FAO was not adopting that approach and we felt that it ought to be, so we found champions within FAO, worked with them, provided additional funding for those champions internally and undertook a campaign internally and externally to provide the evidence through on-the-ground practical experience which then helped the FAO to change some of its policies. That is the sort of work we do in the international system, and obviously we also have a role as member states in influencing the way that the UN and others work.

I think that the key to doing all of these things is building networks and relationships, looking for opportunities, and looking at issues of timeliness, politics and power structures in relation to the way we work with policy, to give us the in-roads to it. And I know that the RAPID programme and others have come up with models that are more effective.

To give you a bit of a case study, I thought I would tell you how we developed our research strategy in DFID. First, I will give you a little bit of the pre-history. We funded research in about 12 different departments. There was a review commission and a report, which looked at the different sectors of research, found that the research had quite a localised impact. It also found that there was a lot of duplication, they were not working together very effectively and the focus should be much more on outcomes than sectors. As a result, we formed a new central research team which brought together all of the different bits of research. We had £75 million worth of commitments that we had to honour at the same time as developing the new strategy. It was formed as part of the overall change within DFID which resulted in the development of the Policy Division. Our role was to produce long-term global public goods, whilst the Policy Division's role was very much to look internally at what DFID was doing. We had quite a short time frame for doing this, so again, timing is very important. We had to get decisions on our new strategy from April to the Secretary of State by Christmas, which is quite a short timeframe, and this is to spend a budget of about £100 million a year.

So how did we do that? There are government guidelines on how to run consultations. One such guideline is that you have to have an open consultation on the web, which we did. We received 600 submissions, 95% of which were from the UK. We had hoped that we would get a few more from other parts of the world. Most of those submissions were from researchers putting in their ideas for their own particular projects, but what we wanted was big picture strategic stuff and we did not get much of that at all.

We ran three meetings with top international academics in different sectors. One was run by the Development Studies Association, focusing on the social side, one was run by the Royal Society for Tropical Medicine on the health side, and one was run by the Tropical Agriculture Association on the agricultural side. Again, they came up with some quite interesting things around research processes and ways of doing things, but not really the kind of big ideas that we were looking for. We then decided that we did not know enough about certain things, so this is where we commissioned the production of some evidence. We did six background papers, one looking at the UK system, one looking at how developing countries' research systems work, one looking at the big international picture in research, one around how we engage better with the private sector and two on communications. One of the communications background papers was looking at how research could be communicated more effectively and one was looking at how we could engage better with policy shapers.

I led the one on communication and policy shaping. I think that one of the critical things for that paper was that, through my previous contacts, I had good relations with a number of people who are leaders in their field in the academic community (many of whom are here today). This meant that I was able, with their support, to bring them in to help me work through some of these issues and produce a robust paper, which provided the evidence, but which also had a one-page summary with three fairly simple recommendations that we could then adopt. That is where these networks are critical and I think the approach really worked. We did do some donor consultation, and we worked with the International Development Research Centre (IDRC) and with Rockefeller to see what they had been doing, but we probably did not do enough of that. We also worked with our own advisory groups internally and our Heads of Profession, but again, we did not get much response from our own people within DFID and, at that point, I think science still did not have a very high profile in DFID.

Then things started to change. Because we were developing a new strategy and there were indications that the strategy would move funds away from the UK and give more funds to developing countries, there was outcry from the UK academic community. They set up various lobbying groups, to our Parliament, to Sir David King, to our Select Committee on Science and Technology. They were concerned about a number of things. One of their concerns was that we were developing this strategy very quickly and that we had not even properly evaluated the past research that we had been funding. They were also concerned about the lack of funding in this area in the UK generally and that now they were seeing this funding moving away.

What happened was that we had a Select Committee enquiry, by the Science and Technology Committee in the UK Parliament, on the role of science and technology in international development policy in the UK. I think this was the first time that we had had a Select Committee enquiry to DFID from anyone other than the International Development Committee, so for us it was interesting. We were slated by the Committee, who criticised our most senior members of staff, and we were not necessarily able to defend ourselves against some of it. I do think that some of it was wrong, but we nonetheless felt a little bit uncomfortable and our Minister came under political pressure. It was at that point, based on evidence and on his increased awareness of the role and importance of science for development, that he decided to double the budget, put out the new research funding framework and recruit a Chief Scientific Advisor to DFID. There were other processes going on internally as well which led to these decisions; it was not just the enquiry. It was a process of internal and external lobbying which had enabled this policy change in DFID.

We then needed to start looking at its implementation. There was a lot of room for improvement. We are now about to embark on developing our next research funding framework to cover the next five years. The principles are right in terms of the way we have been doing things, but one of the biggest problems we had was that we just did not have enough time, or even the mechanisms, to do much developing country consultation, and I think that this time round we have to do much more on that. We probably did not do enough in terms of mapping our strategy against what was happening internationally, in the regions and in developing countries and making sure that we were aligned with others. We probably did not do enough to gain DFID-wide ownership. Now that we have the political support of the Secretary of State and are developing a Science and Innovation Strategy, I hope that when the country offices are developing Poverty Reduction Strategies, they will say that science and research are an important part of that process. We need to evaluate past strategies and we will do something on that. We also need to be much more connected with our key lobby groups and the key players in the UK, those who are likely to shout, and we need to bring them in right at an early stage so that rather than shouting at us, they are actually supporting us.

We do need more big ideas. Donor coordination is something that is key. We need to work out whether or not the international community has an overall strategy in this area and whether we as donors look at divisions of labour or work separately. We need to use our Chief Scientific Advisor, Sir Gordon Conway, as a champion.

I will just touch briefly on one further issue and this is the way in which we have implemented our research. We have a number of different modalities that we can use and ways in which we can fund research and I think this is to DFID's advantage. We commission our own research through open competition but we require at least three developing country institutes to be part of that process. The reason for doing that is that we think it is really important to build capacity at the national level for researchers to engage. The difficulty we have experienced with trying to do this is that they come in as partners at the beginning, but when you start to look at the reality of the partnership, they tend to be very weak partners and do not get much money. We need to do something differently to make sure that developing country research institutes are proper partners as part of that research process. We contribute to multilaterals like the Consultative Group on International Agricultural Research (CGIAR) and we have a role in looking at their overall strategies.

A big new area for us is in product development and partnerships with the private sector, so we put a lot of money into things like the Medicines for Malaria initiative and the international AIDS vaccine initiative. We are finding that things like these are quite important, because they lever a lot more money from the private sector into areas where there is market failure. For example, there is not enough money in malaria sales in developing countries for the big drug companies to get involved, so if we can provide extra resources and help build that market for the drug companies, they are then able to provide us with some of the intellectual property that they develop. There are a lot of new experiences around this and I expect that we will continue to engage more in some of those areas. We are also now investing more in regional programmes, so what we are saying in agriculture, for example, is that we need to put money through the emerging regional agricultural research organisations like ASARECA in East Africa, so that their capacity is built and the funding gaps they have had in the past are being reduced. They are much more likely to be able to influence national policy because they are there on the ground rather than being at a different level. Obviously, we need to look at long-term sustainability, how we eventually pull out from putting money into the national systems and how it can be done more at the national level.

Through the Select Committee inquiry and other processes, we realised that we were not really doing much with the UK Research Councils. The Research Councils do a lot of very good blue skies research that we could utilise in a more applied way. So we are developing a lot more partnerships now with research councils to try to lever in money from other parts of the science system and adapt that science so that it is more usable and applicable to our and developing countries' agendas. We are also working a lot more with other donors and doing joint donor projects. Again, processes are important and I will talk about those again in a moment, but communication, capacity-building, monitoring, evaluation and learning are all really important as well. Again, we probably do not do enough now and should do more.

I want to give you a practical example of how research has made a difference and has influenced policy. We have a wider sustainable agriculture research strategy, which I will not go into in too much detail, but one of the research projects that we funded previously was looking at something called Mile a minute weed. This weed was introduced into India and other parts of Asia as a ground cover, so the idea was that it would grow underneath the crops and stop weeds coming up, but what happened was that it just grew over everything. Once it was introduced, we had the huge problem of what to do about it. There was a ten year research programme that started to look at bio controls and it screened numbers of pathogens from around the world. This research provided the evidence and came up with a rust which would target this particular weed without targeting anything else. So they found a technical solution, which was fine, but then they had the problem of how to get this into use and how to change policy. Somebody decided to hold a workshop with the key policy-makers in this area and brought them together in India to suggest doing some very small-scale trials so that it might be possible to start seeing the impact of this thing (prior to this you could not do trials because India did not have a bio pesticide policy), and they also brought an economist in to show the economic impact of the problem. Once they had done that, the policy-makers agreed to go ahead and start doing some trials, which were very successful, and India now has a bio pesticide policy so that it can release these sort of things. China has now started to adopt it as well. It is a matter of working with different people and different networks, and bringing scientists together for different things, but crucially, it is a matter of providing that evidence. As I think this example shows, it a mixture of working with the national physical scientists and the social scientists.

I will briefly tell you about what we have done in terms of commissioning research and how we have tried to provide incentives to researchers so that their research can be better used by policy-makers. We call it research communication, but it includes policy engagement and influencing and so on. We have basically taken a two-pronged approach. One approach is to try to make research relevant and accessible and I think this is quite important. We have said that any research that DFID commissions directly must have a communications strategy, and that this strategy must be designed at the inception phase (so within the first six months of the project). We have said that if we are not happy with it, the research will not go ahead to the next stage. We have also said that a minimum 10% of the research budget must be spent on communications. By communications we mean engaging with policy-makers, not producing a website (that may be part of it, but it cannot be the only part). We found that researchers were quite resistant to this because they felt that it used money that should be spent on research, so it has been a bit of a battle. We have also produced guidance notes that have drawn on things like the RAPID framework so that researchers have some ideas about how they might do things better. We have provided them with some advice and we have also held learning workshops. For the first time in DFID, we brought together health researchers, agricultural researchers and social science researchers so that they could learn from each other about what they were doing, and we hope that we can continue to do that.

We also had a problem with knowledge management in that, basically, we did not have one easy place where we could find information on all the research we were funding. So we put together a Research for Development website, which is an information portal for all of our research and includes databases of all the research partners we work with. I think there are around 6000 research projects on the site that DFID has funded over the past few years and the research outputs are also all there. The policy-makers are probably not going to do much searching on it, but it is a very good place to find out what we are doing and where, and for people based in a different country office to know which researchers and research institutes are being contracted in their region.

We also have a separate budget for communications projects. Through this, we support things like the Programme for the Enhancement of Research Information (PERI), the International Network for Access to Scientific Publications (INASP) and other online information systems. We are also doing quite a lot of work with the media. We are supporting, for example, the first ever television soap opera in East Africa, Makutano Junction, which is building the capacity of local production teams. The director is the director of a UK soap opera called East Enders and he is providing technical support. Makutano Junction is popular and has enabled us to take messages from our research and turn them into televised stories linked to the characters. We also provide a backup website and a notice that comes up at the end of the programme saying please go to the web address for more information. This means that people can access the actual research outputs. We are trying to take it a stage further and get the programmes to work with the researchers as well as with extension and support services and NGOs at the national level, so that there is a support service for rural people who might watch the programme. We also do quite a lot with the radio and we are doing some interesting stuff with the Panos Institute to try to get science used more regularly in debates, so that when there is a debate on the radio they are using researchers. We are funding the development of the World Federation of Science Journalists, which is trying to support journalists to develop associations so that they can report effectively on science, and we are trying to build that relationship between the science community and the media. There are other projects that we fund and these are all included on our Research for Development website, if you wanted to find out more.

On the other side, we are trying to look at collaboration with other international organisations who are doing the same thing, but we want to play more of an advocacy role, for example by suggesting that other research funders ought to be investing in communication in their research programmes. Learning is quite a big issue and, as I said, the issue of developing capacity is very important. I think that this will probably be a big issue in our new research funding framework. We are not only talking about the capacity of researchers, we are also talking about the capacity of users to use research. We have a big programme called Research Into Use at the moment, focusing on the agricultural side. It is a £37.5 million programme over five years and it is about putting to use the research that was funded under our ten year agricultural programme. The research organisations are actually not too bad; the blockages are in the NGOs, civil society groups, user-groups, the media, the policy environment and intellectual property laws, and the private sector. Working through the issues of how we build capacity there so that evidence is used, will be an important challenge for us.

Finally, what is the future for us in this area? We have to respond to the three priorities in the new White Paper. The White Paper announced again a doubling of DFID's research budget to £220 million a year. There is a big challenge across government and I think it is an issue for those trying to get research into policy. The UK government has said that it is getting rid of 100,000 civil servants, which means a 10% staff cut in DFID. Our team of people managing research at the moment consists of only eight people, so you have eight people managing a £220 million budget and trying to engage with the research community as well. When you are thinking about linking into policy-makers, you have to recognise that there are many more researchers than there are policy-makers and that actually policy-makers are very busy, so three bullet points are great.

We are developing a Science and Innovation strategy and hopefully that will frame science within a broader picture, so we can look at what the real role is of science in development, at where else capacity needs to be built, and at what else our country offices and the international system can do to support that. Obviously, government's capacity to use evidence is a very important part of that. We need to think longer-term so that when we plan our research nationally, we look ahead to where we should be in twenty years time, to what the research world should like, what the policy world should look like, and what we have to do to get there. We need to work in a more joined-up way, so we need to continue our discussions with other donors on the research side but also with the foundations. How do we work more with the foundations? How do we all start to join up? Internally, there is the issue of how DFID becomes a more evidence-based organisation and that will be critical for us if evidence-based policy is going to become the reality.

Thank you.