July 25, 2016
Growing demand for services, reduced fiscal capacity and increased public scrutiny of government spending have led to a spike in the demand for systematic, rigorous evidence in policymaking. But governments generally aren’t set up to do the work of carefully connecting evidence to policy. Research is regularly left to policy analysts far removed from decision-making, and without specialized training. Too often the approach is reverse-engineered: evidence is carefully selected to support the direction governments are already intent on taking.
Enter the evidence institution. Evidence institutions, which can exist within or outside government, possess the technical expertise to review and produce robust policy research, and have the attention of the public and policy-makers. Long the norm in areas such as healthcare, evidence institutions are now popping up in many other policy areas, in response to demand for better evidence to guide decision-making.
For example, the UK government has established a network of eight “What Works” institutions since 2010 that are focused on the generation, transmission and adoption of evidence for a particular policy area – from early childhood development and crime reduction, to healthy aging and local economic development. There are other examples.
Evidence institutions: some leading examples
The UK Behavioural Insights Team brings behavioural economics to the study of policy, having conducted over 150 randomized control trials (RCTs), the current gold standard in policy research. RCTs see researchers test the outcomes of a policy as though in a laboratory – carefully controlling for other external factors to ensure a robust relationship between the policy action and the observed results.
The Washington State Institute for Public Policy (WSIPP) is a non-partisan evidence institution created by the Washington state legislature. At the direction of the legislature, WSIPP conducts rigorous research – especially cost-benefit analysis, which assigns dollar values to the outcomes of policies, and compares them against budgetary and others costs to determine whether citizens ultimately benefit.
Bloomberg’s What Works Cities marshals data and evidence to improve public services in 100 mid-sized US cities. Eligible cities apply to the program for access to a network of experts, who provide technical assistance for sophisticated program evaluation, and the incorporation of data into municipal decision-making.
What Works Centre: A Solution for Canada?
Some recent initiatives notwithstanding, Canada lags behind other countries in producing and using evidence to improve policy-making. But the federal government has recently committed to improving our record on this front. Creating our own evidence institutions – for instance, a Canadian What Works centre or network – would be a major first step.
Canada lags behind other countries in producing evidence to improve policy-making. Creating our own evidence institutions would be a major first step to improving our record.
How to build a What Works Centre for Canada: Key Choices and Considerations
Our analysis of What Works centres in other leading jurisdictions suggests the need for a unique set of skills and capacities: a mix of technical, data-focused professionals, and skilled project-managers who know how to navigate government and provide clear, practical advice.
The key choices confronting Canadian decision-makers in setting up a What Works Centre include:
1] Mandate
Should the centre/network focus on:
- conducting impact studies and program evaluation;
- disseminating the available evidence to decision-makers and the public; and/or
- building the capacity of governments to deliver outcomes-based programming?
2] Evidence Output
Should the centre/network:
- generate new evidence, through experimental and quasi-experimental methods like randomized control trials (for example); and/or
- marshal existing scientific and social scientific evidence, as well as less robust forms of evidence like case studies and best practices?
3] Focus
Should the centre/network be dedicated to:
- a set of specific, highly focused policy areas, subjecting highly politicized questions to world-leading research evaluation (the UK model);
- creating consistent methodological approaches applicable across policy issues (the US model); and/or
- adopting a regional focus (e.g., a Northern What Works Centre with institutional expertise on resource extraction, remoteness, northern infrastructure, and other shared challenges – the What Works Scotland model)?
4] Funding
Should the centre/network be supported:
- solely by governments, either directly or through a trust; or
- primarily through the philanthropic sector, alone or in combination with government funding?
And, should the centre/network be:
- launched with a large endowment;
- rely on project-by-project funding; or
- operate partially on government grants, while also working with government and private clients on a fee-for-service basis?
Policy-makers choosing among these options should be guided by the unique context of Canada. For example, outside of clinical medicine, much of the evaluation evidence produced in Canada does not meet an international standard of rigour. So capacity-building is an essential first step. This could involve partnering with innovative academic centres and experts in particular areas.
And evidence institutions can make a unique contribution in a federation by lining up the various intergovernmental actors behind hard information. Building an effective intergovernmental process is hard enough already. It’s next to impossible when 14 governments are proceeding from 14 separate bodies of evidence.
There’s still the unresolved question of how to ensure new evidence is actually used by policy-makers. We know that there’s a supply problem: governments don’t have access to enough good evidence. It’s possible there’s a demand problem too – that governments aren’t making a sincere effort to consult the available evidence. The international experience suggests that a What Works Centre could significantly enhance our governments’ ability to deliver evidence-informed decisions, programs and service enhancements. But real success will require leadership and buy-in from the public service.
More related to this topic
Authors
Michael Morden
Reuven Shlozberg
Release Date
July 25, 2016
