5. Conclusion
This closing chapter draws the lessons of the analysis of the skills and other contributing factors to evidence-informed policy-making. It offers a number of recommendations to make the use of evidence more effective, including the need to be aware of the local and political context, the need to address the full range of skills and capacities. The recommendations also highlight the institutional and organisational structures and systems that enable effective use of evidence, as well as the role of strategic leadership and the need to embed evaluation from the beginning to inform the implementation process. The chapter also highlights potential areas for future work, including a professional development framework, as well as the need to address the impact of cognitive and motivational aspects of capacity building.
Increasing governments’ capacity for an evidence-informed approach to policy-making that is fully able to make use of policy evaluation within the public sector, is a critical part of fostering good public governance. Based on the evidence that was gathered, this report calls for investing in individual skills for the use of evidence by senior policy-makers and for building capacity for evidence-informed policy-making at a systemic and organisational level.
The goal is to enable agile and responsive government, which is well equipped to address complex and at times “wicked” policy challenges, in a shifting political environment, driven by short-term political pressures and conflicting voices. Evidence has a critical role to play in responding to these challenges, by improving the capacity of government to shape effective public policies and deliver quality, responsiveness and accessibility of public services. Evidence can play a role throughout the key stages of the policy cycle and is increasingly recognised as a critical part of good governance.
Despite the potential for policies to be based on evidence, an effective connection between the supply and the demand for evidence in the policy-making process remains often elusive. Many governments lack the necessary infrastructure to build effective connections between evidence and decision-making. Although civil servants may have access to evidence and acknowledge the importance of using it, many do not use it systematically in crafting policy analysis. Furthermore, policy-makers experience a range of other barriers to accessing timely and relevant evidence. While a single solution and tool may not exist for each of these challenges, there is value for countries in building capacity for policy-making at the systemic and strategic level.
This report presents the interventions; tools and strategies governments can use to build their capacity for an evidence-informed approach that fully leverages the value of evidence for policy-making. It also leverages a skills framework that was jointly elaborated with the EU/JRC. These include: diagnostic tools to understand the range of existing capacities and ensuring that interventions are well matched to governments’ needs; initiatives designed to increase policymakers’ ability to access and obtain evidence; initiatives to improve individual policymaker’s capacity to use evidence; mentoring to provide personalised guidance in relation to ‘real-world’ application of knowledge, and strategies for promoting interaction and engagement between researchers and policy-makers.
The use of evidence is also intimately linked to organisational structures and systems. Improving organisational capacity includes a range of strategies such as improving organisational infrastructure; improving organisational tools, resources and processes; workforce development; and establishing strategic units to support an evidence-informed approach.
In addition to highlighting good practices for enhancing the collective skill set in the public sector, the report offers a framework that countries can use to identify and select interventions, tools and strategies to build their capacity for an evidence-informed approach. The following five key conclusions are designed to maximise the value countries can expect in using this framework:
1. Capacity building initiatives need to be aware of the local political and institutional context of research use.
Without an understanding of how policy-makers engage with research evidence and how they integrate it with other forms of input into the policy-making process, capacity-building initiatives risk being poorly aligned to the local context and culture. In reality, many initiatives have insufficient knowledge of the local practice and context that they are trying to address. Creating capacity-building initiatives that reflect the local needs and context therefore requires generating an understanding of the messy reality of how actual policy-making occurs and what are the opportunities for evidence to play a role. For example, in a government that has civil servants who have all the capabilities to use evidence, but not the motivation, nor opportunity to do so, implementing a generic skills training exercise in isolation is unlikely to effective. The diagnostic tools identified in this report therefore have an important role to play in helping governments understand their current context of evidence use and what range of strategies are necessary to improve their capacity for EIPM. This can be particularly important as governments are facing citizens’ concerns and a lack of trust in public institutions.
2. Capacity building initiatives need to address the full range of skills and capacities that influence the use of evidence, including skills for understanding, obtaining, interrogating and assessing, using and applying evidence, as well as engaging with stakeholders and evaluating success.
Developing capacity for the use of evidence requires consideration of the current capacities within the system, spanning the individual and the organisational levels. A key first step for policy organisations, which are unclear about what their current capacities, are: first, to gather information on the range of current capacities; second, to foster the desire for change; and third, to identify the barriers and facilitators of evidence use within the system. Following a gap analysis, organisations can use the skills framework of this report to identify the right kind of skills and leverage some of the examples that are offered.
3. Institutional and organisational structures and systems enable the effective use of evidence – without addressing these, change initiatives are unlikely to succeed.
Building individual skills and capacity are critical components of strategies to improve the use of evidence. However, a sole focus on individuals limits the potential to engender long-term and system-wide change, especially in the context of the rotation of employees experienced in the civil service, whether as a result of regular rotations or related to a change of government. This means that building the civil service’s capacity to improve the use of evidence necessitates consideration of the institutional elements that can support this. This report has reviewed a range of organisational strategies that show promise, including: strengthening organisational tools, resources and processes; making investments in basic infrastructure; and establishing strategic units to support an evidence-informed approach. Mandates, legislation and regulation are also important tools to facilitate the use of evidence.
4. Strategic leadership is critical to drive the organisational change necessary for improved evidence-informed policy-making.
Strategic and committed leadership is a crucial driver for the change needed to embed an evidence-informed approach to policy-making. Public sector senior civil leadership programmes can help to support senior leaders to face the current challenges of public administration and enable leadership of the evidence agenda. Using high profile positions as a catalyst is another strategy for creating the necessary momentum for driving change in policy settings. These can include senior positions such as Chief Economists, Chief Information Officers, Chief Evaluation Officers and Chief Science Advisors. An evidence-informed approach can also be leveraged through performance-driven approaches to resource allocation. For example, accountability tools can be used to incentivise EIPM by the senior civil service. This can then help to promote the use of evidence throughout public sector organisations: if the senior civil service are held to account for the quality of the evidence base they use to make policy proposals and decisions with, then they will implement measures to ensure that the rest of the organisation is incentivised to use evidence, such as by including evidence use in the civil service competency framework.
5. Capacity building initiatives should embed evaluation from the beginning to inform the implementation process and support continuous learning and improvement.
Evaluating the impact of capacity building interventions should be a priority for governments embarking on initiatives. This is critical to ensure governments do not waste resources on ineffective or inefficient interventions. Despite the increased interest in stimulating demand for evidence and a number of reviews of the effectiveness of different initiatives to do so (Langer, Tripney and Gough, 2016[1]; Haynes et al., 2018[2]), there remains a paucity of evaluations of the impact of interventions. This knowledge gap is especially noticeable in relation to initiatives targeting organisational processes, resources and tools – and so building in rigorous evaluation should be a priority.
6. Capacity building initiatives need to be embedded within organisational structures and strategies to enable sustainability and long-term change.
Evidence-informed policy-making is unlikely to become integrated into the ‘business as usual’ operation of policy organisations as a result of short term and short-lived initiatives. There is a need for continuous involvement and exposure to capacity building initiatives by embedding these into existing work and organisational structures and systems. Greater consideration of these opportunities for organisational learning should reduce the risk that initiatives get ‘washed out’ after an initial period of enthusiasm. Strategies to promote sustainability can include legislative approaches as well as departmental evidence strategies. A consistent message across the reviewed initiatives is that increasing capacity for evidence use is a long term and evolving process. This means often starting with manageable initiatives and building from there, whilst being realistic about when expected results will materialise.
The analysis contained in this report can offer opportunities for follow up work:
Developing a continuous professional development framework: The framework would also need to specify which individuals within a given context require which skills in order to contribute towards an evidence-informed approach. For example, elected officials can play an important role in creating demand for evidence and creating a culture of evidence use within Ministries. This continuous professional development could be offered at various stages, including senior and junior level. The Senior Civil Service also plays a critical role in policy development, and may remain in post longer than elected officials. Therefore, supporting the Senior Civil Service to build their capacity can help deliver results in the long term. Alternatively, investing in more junior levels may also represent an investment in the future, and junior members may also have significant responsibility in shaping and drafting policy proposals from the start at the inception phase.
Collecting examples of initiatives from a wider range of policy areas and countries. This report benefited from a significant range of expertise and grey literature, as well as engagement with OECD stakeholders. Nevertheless, it could be further enriched by more in-depth country examples. Delegates will be invited to submit any additional information to enrich the findings and analysis of the report, particularly across linguistic boundaries for countries where the information is more difficult to identify.
Further addressing the impact of cognitive and motivational aspects of capacity building. One takeaway from the reviews of interventions included in this report e.g. (Langer, Tripney and Gough, 2016[1]; Haynes et al., 2018[2]) is that interventions to improve evidence-informed policy-making need to be based on sound programme logic and the best available evidence. While this report helps to consolidate policy-making process through improved facts and policy evaluations, addressing more fully the cognitive dimension might be helpful in the future. The role of self-interest can also be very important. The role of memory bias, information asymmetries, aversion to risk and bias in perception might play an important role in skewing policy outcomes. It might be useful to create opportunities for the policy-making process to become aware of the potential impact of such biases to at least ensure that decisions are taken in a full information environment. Some of the work conducted by the EU JRC as part of its Enlightenment 2.0 project1 and the pilot project "Science meets Parliaments / Science meets Regions" might also be very relevant in this respect. Similarly, there is further scope to investigate the motivational and cultural factors necessary to generate a culture of EIPM in the civil service, which could benefit from further OECD work on public employment and management.
References
[2] Haynes, A. et al. (2018), “What can we learn from interventions that aim to increase policy-makers’ capacity to use research? A realist scoping review”, Health Research Policy and Systems, Vol. 16/1, p. 31, https://doi.org/10.1186/s12961-018-0277-1.
[1] Langer, L., J. Tripney and D. Gough (2016), The science of using science: researching the use of Research evidence in decision-making..
This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.
https://doi.org/10.1787/86331250-en
© OECD 2020
The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.
