I spent a fascinating afternoon earlier this week at a roundtable hosted by NESTA (the National Endowment for Science and the Arts) to discuss What Works Now – Evidence Informed Policy and Practice (Policy Press 2019), a book edited by Annette Boaz, Huw Davies, Alec Fraser and Sandra Nutley. The collection of essays build on the earlier What Works? (Policy Press, 2000), and brings together pieces that look comprehensively at how evidence is used to inform practice and policy making across different sectors and in different countries.
I won’t attempt to summarise or review the book (except to say it is well worth getting hold of a copy, as it is as readable as it is comprehensive), or the incredibly rich discussion chaired by Geoff Mulgan. However to highlight a few thoughts and challenges which came out of the discussion which I think are relevant to those of us involved in connecting academic with research with policymakers.
First, that the supply side has transformed over the last twenty years. One can debate whether decision makers are more or less inclined to take evidence informed decisions than they were at the beginning of the century (and I couldn’t possibly comment), but, for example, the development of “what works” centres, and the incentives created by the role of impact in REF case studies have had a huge effect.
This isn’t unproblematic. What work centres risk entrenching orthodoxies and fetishizing hierarchies of evidence rather than being vehicles for what might be useful for policymakers and practitioners to answer this question now. The REF impact model doesn’t work perfectly for policy impact – there are poor returns for high quality synthesis which is often exactly what policymakers need, and the line from a particular piece of research to significant policy impact might be far from linear, especially where the policy problem hasn’t been researched before.
Of course better to have these problems than not, but could the incentives work better?
And there is a tension for us, as knowledge brokers. On the one hand our role is to get individual pieces of policy relevant research under the noses of policymakers, because case studies pay the bills; but on the other our usefulness to policymakers is as an intelligent gateway that manages the flow of information.
Second, that the demand side has had less focus and critical analysis than supply. There has been a tendency, perhaps because the people thinking about the problem have been the suppliers, to see the challenge as simply one of supply – that the reason decisions haven’t been evidence informed is because the evidence wasn’t accessible, or wasn’t in a usable form, or that the demand side problem was that policymakers didn’t have the right skills to make use of the supply. Because if they did, then surely they would use it, right?
These things were and are true – evidence could be more accessible, usable and policymakers don’t necessarily have the skills to make use of it. But good evidence is a necessary but not sufficient condition for a good policy decision (a number of essays in the collection reflect this, but Professor Paul Cairney’s does so with particular clarity). Policy decisions are synthetic judgements that bring together evidence about the issue in hand, competing priorities, values, public attitudes and political calculations, made by people with limited bandwidth and time.
So understanding how decisions are made on the demand side, ought to be playing a bigger role in shaping the way we supply it.
And finally: to what extent is the work we do as “knowledge brokers” informed by those looking at evidence and policy from an academic standpoint? Is there a role for UPEN to bring these researchers together with the policymakers on the demand side to develop a deeper understanding of how evidence informs decision making? And what is it that we should be saying to UKRI and others about better to incentivise policy impact in the future? I don’t know what the answers are, but What Works Now is a great place to start thinking about what they might be.
Stephen Meek is Director of the Institute of Policy and Engagement at the University of Nottingham and Chair of UPEN for the 2019-2020 academic year.
Author: Stephen Meek, Director of the Institute of Policy and Engagement at the University of Nottingham and Chair of UPEN for the 2019-2020 academic year
