Evidence-based policy making in Wales

The paper that I produced before the last assembly election was very much aimed at setting up the expert versus participative discussion and came down on the side of a fairly classic civil servants' middle-of-the-road position in favour of informed participation.

Whilst I am not going to go back through that paper, I want to start from that expert/participative discussion, as it is one of the issues which is relevant to civil service reform agendas. There is an attitude, mainly seen in retired Cabinet Secretaries, of wishing that they had an even bigger towel over their heads, so that people would come and talk to senior civil servants about policy-making rather than talking to all these other people that they talk to. I think that this is quite a dangerous use of this debate.

In modern policy making, we are dealing with huge complexity of influences and actors. So I want to build from the discussion in my older circulated paper and talk about policy-making as a form of network activity. Previous speakers have hinted at this by talking about the importance of social time and the lack of thinking space. One of the big issues is how we create that space and that networking, and what part research has in all of that.

In Wales, in many ways we have had to address this issue straight on because we did not really have a policy culture. We did not have an extensive civil society engagement with Wales issues as such and we did not have a research base on Wales or people working on Wales issues. Before devolution we had England and Wales statistics, so that it was very difficult to tell anything about Wales from the sort of data and policy-making that we had before devolution. We were very clear that part of nation-building and the devolution settlement would be actually creating those sort of networks and expertise. The fact that we did not have too much policy baggage in terms of 'how we do things round here' has been quite useful in terms of challenging people and bringing new things in.

I have a man working with me at the moment on secondment from Oxfam and he was shocked on entering into a bureaucratic structure by how difficult it is do anything. There is a perception of inertia around a strong civil service culture which can mean that we pursue diversity only to then make everyone the same. In fact, there is not one civil service culture but many, and different departments can have quite strong, independent cultures. These departmental cultures and inertia in the way we do things are often built up by a whole series of folk myths. For example, I came across a folk myth that Peter Walker had not liked an evaluation report because he did not want to be told that what he was doing did not work. That myth was still ricocheting around the organisation fifteen years later.

A bureaucracy works through system and most of our systems are designed to control finance, they are not designed to control policy. There are very few policy systems. Departments like the Foreign and Commonwealth Office have policy clearing because they need to make sure that policies fit together, but elsewhere this is quite limited. There is still a sort of glorious amateurism to it all and this breeds a real lack of learning. We talk about knowledge bases, but there are no operational strategic knowledge bases across Whitehall (to my knowledge!) and little effective knowledge management. There is a whole lack of infrastructure for learning within government and this is a real challenge for all of us trying to work in this area. Perhaps most of all, there is shortage of people asking 'why' questions - questions about what is the logical framework and why we are doing this. Perhaps the most useful questions I have been able to ask anyone are precisely those: why are you doing this, where is this coming from and what evidence is it rooted in? Our recently retired Chief Economist used to have the phrase 'don't just do something, stand there', to remind his colleagues that occasionally it was worth thinking about what they were doing before they actually did it. I think that is quite a nice reminder.

So there are a set of issues about how a bureaucracy works which we need to be aware of and think about how to engage with. The fact that there are finance systems which work and the fact that there are cultures means that a conscious decision needs to be made to put in systems which use and drive these things and shape these cultures differently.

Similarly on the political side, I think we need to see how we can turn the often cited problems into opportunities. We have talked about issues relating to political timescale, which is always important. One thing we have not touched on is a set of issues around prioritisation. Often, a lot of what departments are doing is not very effective and this is almost conscious - departments know that they cannot do everything but they have to be seen to be doing everything. It is important to know which things we really want to achieve and see through to a result. Understanding what is just activity and what is change intervention gives us a much clearer sense of what is going on in government. I am reminded of my social worker father who used to wonder whether he was an agent of social change or social control.

One other area of risk is that the attraction of the new is greater than the attraction of things which have been tried in the past and have achieved some impact. There are numerous new grant programmes which are running as if the previous set of grant programmes had never existed. Grant programmes are always piloting new things, rarely learning from previous initiatives. They are always seeking another three years of short-term funding.

I was very struck by an ex-strategy director colleague, from a Department which shall remain nameless, who said it was very difficult to develop strategy when the government's policy was to do what works. There is a whole set of things bundled up within that, including locating the values within a policy, the context within which you are defining what works, and the complexity of government. It was interesting to see the Prime Minister turn back towards looking at strategy and not just targets a few years ago. He introduced the idea of five-year strategies for Departments, specifically saying that delivery was more complicated than he had thought. A target could be achieved in one area but then something goes wrong somewhere else and these areas needed to be looked at together in light of what was trying to be achieved.

In Wales, we are on the case with trying to tackle a few of these issues. We have an overall published strategy for the Government's term (Wales: A Better Country) on which we can hang policy-testing and appraisal and which means that we do not keep reinventing policies. We are appointing a chief social researcher and have created an office for social research, which effectively brings together into a fairly loose alliance the various research capacities which we have. We have also developed a specific research development pack for people who want to develop through the research grades. That is a sister division to mine within the Strategy and Communications Department.. The economists are also working within this Department, and we have good links to the statisticians who are based in Business services. There has been a lot of work to build the core statistics base which was horrendously low when we took it over. In capacity terms I am beginning to feel that we are reaching the kind of critical mass of support and influence that we need.

I never used to be keen on systems but bureaucracies need systems, it is the way they operate, so I am having to become quite interested in them. Every Welsh assembly Government Department now has a research and evaluation programme which is signed-off every year and is then linked back into operational planning so that it becomes part of what the department is actually doing.

We consciously carried out a values-led spending review last time round, looking at how each particular activity was contributing to the government's top level objectives from Wales: A better Country. We have taken a lot of those principles into the new operational planning and budgeting round this year, in which for the first time we will be asking for every budget line to state what the outputs will be. We know that outputs do not relate to outcomes but, at the very least, it means that we know the outputs we are buying and why.

We use the Freedom of Information Act very consciously and we are a bit ahead on this. For the last two years now, against every ministerial decision we have published on the internet the evidence and reason behind that decision. It was fascinating getting behind some of the tacit cultural issues in preparing people for that. In particular, I remember a colleague saying, 'but what about the things you just know?' As a definition of tacit knowledge that works quite well. In this particular case he was actually talking about making recommendations on appointments, which was a particularly dangerous place to use tacit knowledge. In effect, the Freedom of Information Act has forced people to articulate, in a way that had not previously been required, why it is that they are recommending what they are recommending. In this respect, it has been really quite useful.

We have also developed what we call the Policy Gateway, which every major policy has to go through. It is effectively a tool which asks whether the connections with other relevant policy aims have been thought through. It has been quite a useful joining up tool, but it also operates as a sift for other more formal appraisals.

Policy learning though comparative work has also been important for us. I have been doing quite a lot of international networking around sustainable development over the last three or four years, which has been very useful. Some of the things which we want to do in Wales are not the same as in English policy and there are no ready comparators, so certainly from my point of view, tapping into some of the developing country perspectives has been really helpful in challenging some of the core assumptions. I am also increasingly looking at the experience of Scandinavia and some of their theories about issues relating to small country networking and change. We are taking some of their theories into the work which we are doing around our public service programme, which is very consciously communitaire, not target-based and not competition-based, reflecting the fact that we are a small, rural and relatively poorer country than most of England. Our public services work, Making the Connections has taken this values-based position in terms of where we want to take public services which is quite broadly defined, so that we are now trying to put in the underpinnings of how to make it work. The Scandinavians will probably be quite useful to us in doing this. Similarly, our work on the Wales Spatial Plan, People, Places, Futures has been a very open-ended and participative process, drawing heavily on a European approach to spatial planning. We have been pushing the boundaries of current practice in much of this work, whilst trying to draw on the experiences of other countries to ensure we are not running up a blind ally.

I will pick up one particular recent area where I think that the policy-research interface has been particularly tricky. This is in the area of equality policy and particularly in mainstreaming equality. We have faced a lot of competing models and approaches amongst those we have engaged with and there has been a real difficulty in dialogue, particularly around understanding how we work and in talking about those sorts of change processes which are involved in mainstreaming equality. This is one of the areas where we really need to work together to understand how a bureaucracy works, what the levers of change are in a complex bureaucracy, where we need to influence and how to work within these kinds of structures. We have been having some real troubles with fitting the theoretical models on equality mainstreaming (which are not consistent themselves) together with some of the realities of the way we operate in practice, both in the individual pieces of commissioned research and in some of the dialogue we have had.

I could talk about other examples, but I realise that time is short. To close, then I will mention just a few things which I think are really key. First, I still think informed participation is central. If you can create enough space to bring people together to discuss a particular issue, the benefits can be great: it can be enjoyable, it is stimulating for the people who come; and gives people the opportunity to go back with a few new ideas. So I hate the language of research 'users' - this is not the language of networks and thinking spaces.

Second, what was said in my circulated paper about context and relevance is very true for us in Wales, especially where we have very little information about Wales, where we have a distinct policy approach on some of the key issues in relation to public services and where we have distinct legal and bureaucratic structures. Policy and theory are not value-free; they always come with values and always come with context. Evidence-based policy making must reflect this.

Perhaps above all, when I think about where we need to take things, it is to address this lack of knowledge management and the lack of a knowledge-base within government. Without these, anything we do in government at the moment will work while it is working and be gone next year. The more I think about it, the more this seems to be the gaping hole in the evidence-based approach.

Comments and Questions from the floor included:

• It was pointed out that there is a focus on the research and policy inter-phase but not enough discussion about the research-practice process of the corporate sector. Researchers and firms often have different research timeframes than that of the policy process and cannot wait for it. Unfortunately, the policy agenda is still often placed at the centre of the process and all other actors need to rearrange themselves around it.
  
• A question of clarification was made on the meaning of 'backfilling with evidence'. Mathew Quinn explained that in the case of Wales, a choice has been made to follow a value driven approach to policymaking. They now have to develop the necessary evidence to defend it against an efficiency based model and to develop the desired interventions: use evidence to get the method right.
  
• Mathew Quinn observed that building up knowledge management capacities is crucial but costly and takes time. The use of ICT can provide significant solutions but requires additional resources than often available.
  
• Phil Davies agreed that there is a need to build up more systematic reviews in order to improve the supply of evidence and to increase the level of demand for evidence. He mentioned that methods and tools of knowledge management were neglected during the public sector reforms in the UK and are now inadequate.
  
• Participants considered that one of the greatest challenges is how to get people who have limited time to use evidence. DFID, for example, is being asked to increase the productivity of its staff, and the first thing that will fall off the agenda will be maintaining the support structure for the networks that link evidence and policy.