Annex A. Data and methodology
This comparative analysis of policy evaluation builds on the data collected though the 018 OECD Survey on Policy Evaluation (hereafter, the “OECD Survey”). The OECD Survey is a direct response to the request to collect better data in the area of evidence informed policy making by the Public Governance Committee, in the context of the development of a Policy Framework on Sound Public Governance. (see https://www.oecd.org/governance/policy-framework-on-sound-public-governance). This Framework was derived from the aim to support policymakers by consolidating an integrated vision and coherent narrative of the main elements of sound public governance. The information on policy evaluation practices was gathered as part of the effort to inform the development of the Framework chapters on policy formulation, implementation and evaluation.
While the report mainly draws on the data gathered through this OECD Survey on Policy Evaluation, it also uses related data from the OECD performance budgeting survey (https://www.oecd.org/governance/budgeting/internationalbudgetpracticesandproceduresdatabase.htm) and also from the work on measuring regulatory governance, under the Regulatory Policy Committee (http://www.oecd.org/gov/regulatory-policy/measuring-regulatory-performance.htm). It also relies on comparative data on the Centres of Government (COG) from the “Centre Stage” Publications and related COG surveys (see https://www.oecd.org/gov/centre-stage-2.pdf).
After a consultation and approval by the Public Governance Committee Delegates, and a piloting with a few volunteer countries, the survey was officially launched in mid 2018, with a range of countries involved in the OECD work on public governance. The process of data collection ended finally in mid 2019, with data being available for 42 countries. Countries were allowed to update the results to show any reforms in place and implemented as of mid 2019.
The survey is structured in two parts:
Policy evaluation across government.
Policy evaluation at the sector/thematic level.
In the first part of the survey, respondents were asked to provide information and data from a whole of government perspective on policy evaluation. The questionnaire was sent to the main institution responsible for the promotion, coordination and implementation of policy evaluation across government and answered by civil servants at director/assistant secretary level. The current report mainly draws from this main part of the survey.
The survey had a second component at sector and thematic level where respondents from the ministries of health and public sector reform were asked to provide information and data about the governance of policy evaluation in their institution. The current report has done a preliminary use of the data for the purpose of highlighting variations between main practices and some of the sectoral practices.
Survey Responses
Data were received from 42 countries (including 35 OECD member countries with the exception of Luxembourg). More specifically, 42 countries replied to the first part of the OECD Survey, and 33 countries’ ministries of health or equivalent institution and 31 ministries of public sector reform or equivalent institution replied to the second part of the survey. In terms of the non-members at the time of running the survey, responses were received from Argentina, Brazil, Bulgaria, Colombia, Costa Rica, Kazakhstan, and Romania.
The OECD Survey questionnaire is available at the link below: www.oecd.org/gov/institutionalisation-of-policy-evaluation-questionnaire.pdf.
Throughout the report, data are presented according to countries, which have been abbreviated according to the official ISO country codes.
Process for data collection and validation
The survey was distributed through the network of the Public Governance Committee Delegates, who were invited to designate a focal point at national level. The Secretariat carefully reviewed all the survey submissions, with additional requests for information on approaches, clarifications and potential good practices. A process of validation and mutual dialogue with each of the responding country did follow. Moreover, an experts’ meeting was organised in September 2019 where the survey respondents were invited to the OECD to discuss preliminary results and given a final opportunity to validate their country’s data. (See http://www.oecd.org/gov/agenda%20expert%20group%20meeting%20final.pdf).
The survey questions were designed to be responded to in a quantifiable manner, such as yes or no options and multiple-choice questions. Whenever possible, the OECD Survey offered space for countries to give additional information. While the survey can provide aggregate comparative information, additional boxes and qualitative analysis were added to reflex the complex processes of policy evaluation and evidence informed policy making.
This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.
https://doi.org/10.1787/89b1577d-en
© OECD 2020
The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.