Executive summary
This report looks at how countries can mobilise evidence for good governance, taking stock of standards that OECD countries may use to support the design, implementation, and evaluation of public interventions. This is part of a broad agenda on fostering evidence-informed policy-making that is shared by many countries. Existing knowledge brokers, such as the UK What Works Centres, the US clearinghouses or other government bodies specialised in providing quantitative evidence, have made extensive progress in developing standards of evidence for many areas where the public sector can make a decisive difference in terms of inclusive growth, such as health, education, employment, labour and social policies. Yet, users report being confused at the differences among the standards used by various organisations to determine what counts as evidence and how evidence is to be employed in policy-making. Bringing coherence to existing approaches would offer governments more operational options for using evidence.
A first goal of this report is to increase coherence among existing approaches, and the thus raise confidence in them. The existence of robust standards of evidence is fundamental for convincingly communicating on the quality of evidence brought to bear in the evaluation of policies and programmes. The current review is designed to serve central government agencies and knowledge brokers aiming to improve public sector effectiveness by strengthening the evidence and decision-making interface.
Evidence in this report is not limited to data and statistics; rather, it refers to a broader range of analysis that can also include evaluations, scientific research, and other facts derived from observation or social sciences. Knowledge brokers have an important role to play in bridging the divide between producers of raw evidence and policy makers. They carry out a range of functions, including that of ensuring the appropriate evidence makes its way to policymakers in a timely, attractive, relevant, and accessible manner. This evidence has to feed into the policy making cycle for designing, implementing and evaluating policies at the right time and the right format.
The report maps a wide range of existing standards of evidence, related to the quality of content, including over 50 existing approaches. These standards are derived from selected existing and varying practices currently in use by OECD countries and international research bodies. The report synthesises these existing standards through a stocktaking exercise, articulating the results across seven key dimensions: the role of evidence synthesis; the theory of change and logic underpinning an intervention; the design and development of policies and programmes; the efficacy, effectiveness, and cost effectiveness of an intervention as well as how standards for evidence are related to the implementation and scale up of public interventions.
The consideration of standards of evidence, while necessary, is not sufficient for an evidence-informed approach that maintains trust in policy making processes for the design, implementation, and evaluation of public interventions and policies. Maintaining trust requires an investment in good governance, so that citizens and the broader public can infer that the policy process is designed as a way for the government to be able to achieve the best possible outcome taking into account existing constraints. Evidence is but one contributing element to policy making. Policy makers must also weigh up other considerations, such as ethics, economics, and political considerations. Policy makers need to think critically about what evidence is needed in the context of a decision, as well as whose voices, interests, and values need to be taken into account.
As a result, the report also presents a set of principles for the management of evidence used across OECD countries. These principles focus on the process and the conditions for good use through which evidence is brought to bear as part of any process aimed at ensuring trust in decision making. The seven principles identified include: whether evidence is appropriate for the policy concern, accountability of the policy advisory system through which evidence is brought to bear, contestability (i.e. that evidence is open to questioning and has been subject to quality control), public representation in decision making (through public engagement), transparency in the use of evidence, and building evidence through emerging technologies and mobilising data in some ethical ways. The report maps 29 approaches that cover one or more such principles.
Altogether, the combined analysis seeks to improve the quality and use of “standards” and “principles” of evidence for government agencies through a comprehensive mapping and stocktaking approach. While the report is not prescriptive as such, for each of the 14 principles that have been highlighted, it offers a set of questions to help frame the main concerns that should be raised to the attention of government agencies and knowledge brokers dealing with evidence to inform the policy-making cycle. This analysis would aim to support better use of evidence by government agencies.
The findings of this report can strengthen the evidence and decision-making interface, bringing greater understanding and support to the role of knowledge brokerage organisations in their efforts to improve public sector effectiveness. The use of evidence should not be limited to informing the policy-making processes. Evidence is also critical to identifying positive and negative consequences of public interventions, while quality evidence can contribute to learning how to maximise benefits and minimise damages of public interventions. Evidence matters for understanding how existing programmes can be evaluated and improved. This “learning” dimension is an important part of the process of evaluating policy choices and developing systematic approaches to improving public policy and improving good governance. This report highlights the benefits of and the need for global collaboration in terms of data and evidence sharing to support both scientific and government processes. The mapping offered by the report can help further strengthen evidence informed policy making in the future and support increasing citizen trust in the decision-making process.