In this post, Raj Patel outlines three types of challenging problems and argues that longitudinal studies can play an important role in addressing them.

Many social policies can be difficult to formulate and implement well because they depend on a level of knowledge, mobilisation, governance and policy control that is rarely possible in real life, particularly as society become more heterogenous and diverse. For more complex problems, it may not be entirely clear what is driving a problem, and feedback loops make it difficult to disentangle cause and effect.

This complexity and diversity raise challenges. The UK is home to a rich set of longitudinal and cohort studies and a world leader in funding such studies - so could they play a bigger role in tackling challenging social policy issues? Specifically, three major challenges are considered here: silo working, the challenges of developing preventative policies and short termism in decision making.

Cross-cutting policies

The first is the challenge of tackling cross-cutting issues and breaking down boundaries between different disciplines and policy areas. This is not new. Social or business problems are rarely confined to a single research discipline – or a single government department.

The Nurse Review (2015) called for more emphasis on multi-disciplinary research, and a recent survey of civil servants on how they could deliver better public services found that the top issue was cross-team collaboration. Given the emphasis on policy delivery, when it comes to service delivery making sure each part of government’s ‘supply chain’ is not overwhelmed by policy initiatives is critical. Public health is a classic example of an issue that can’t be addressed effectively by a single agency or a single department.

There is also a culture of ‘working in silos’ with data across businesses, government departments, civil society and individuals. Public (and private) administrative data offer the prospect of enabling researchers to access large amounts of information at a relatively low cost. This data is created when people interact with public services and can be linked with rich survey data.

Prevention policy

There is ample evidence to show the scarring effects of life events such as adverse childhood experiences, persistent low income, family break-down or mental illness. Policy interventions can feel like a ‘whack-a-mole’, where a problem addressed in one area pops up elsewhere in a different guise at a later date.

A primary function of government is to help reduce risks faced by society and business. The idea of early action to prevent or mitigate harm has been gaining traction, but progress has been slow, particularly in tackling long-standing inequalities. With prevention, action is usually long-term, more complex to control, less visible, harder to measure and more likely to involve controversial choices, making it more difficult to make progress without building coalitions.

Short-termism in decision making

Then, there is the related but distinct issue of short-termism in policymaking. Policymakers usually make decisions in a complex environment with limited time for reflection. Ministers want to demonstrate progress quickly and are usually rewarded for spending public funds on today’s visible problems rather than reducing future risks. Short-term and symbolic policies will be the right approach for particular situations, but good government can’t simply be about better firefighting – it needs to learn to prevent fires.

Social mobility, poverty, public health, climate change, housing, social care and regional disparities are long-term challenges which have to be addressed across parliaments. The authors of the Institute for Government’s Making policy stick report say that staying focused on long-term policy goals has been a weakness of government.

Stepping up

Longitudinal studies play an important role in research and policy because of unique data they offer. They collect structured information on children, adults and households, often across multiple generations, and over many years and decades. Their explanatory and predictive power comes from their design, tracking the same people over time, rather than providing a snapshot of people’s lives at a point in time.

These studies face many common challenges with other types of social science when it comes to knowledge exchange and impact generation, but how can they play a bigger role in ‘moving the policy dial’? This explored in a new report, Transforming social policies: insights, ideas and challenges for mobilising data and evidence.

In the context of ‘building back better’ from the pandemic, the report argues that this will require more creativity in how researchers go about generating impact. The Transforming Social Policies report, published by Understanding Society, aims to help readers understand the policy process, and proposes new forms of collaboration: to change the way social problems are thought about; support bottom-up social innovation; and strengthen the business case for early action to pre-empt welfare needs.

Raj Patel is the Associate Director of Policy at Understanding Society, Institute for Social and Economic Research, University of Essex.