Assessing regulators’ performance
Economic regulators play an important role in sectors that deliver essential services to citizens and the economy. Robust performance assessment helps regulators understand where to adjust their approach to improve outcomes. According to the OECD Best Practice Principles on the Governance of Regulators, a well-designed performance framework serves multiple goals: demonstrating the effectiveness of the regulator, building confidence in the regulatory system and driving improvements (OECD, 2014). Performance assessment is a critical ingredient for maintaining accountability and fostering transparency, and public bodies are often required to report on results and enable scrutiny of their performance. Data on the performance of both the regulator and the regulated sector are an important ingredient of economic regulators’ performance assessment frameworks. The results can also be part of organisational learning, providing inputs into decision making.
Results from the 2018 OECD Indicators on the Governance of Sector Regulators show that energy, e-communications, rail transport, air transport and water sector regulators in OECD countries could strengthen their reporting in some categories of information about their own performance, in order to produce a more holistic view of performance and to enhance accountability. For example, some regulators collect (78%) and publish (57%) performance information about the quality of the regulatory process (Table 7.12). This type of organisational performance information should be complemented by outward-looking performance information assessing the performance of the sector and final outcomes for customers. Many regulators do not collect or publish information in the other categories relevant to organisational performance and efficiency: compliance with legal obligations, organisational governance, and the operational service delivery of the regulator (Online Figure G.31). The most commonly collected and reported information across sectors and countries is on the performance of the regulated sector and the financial performance of the regulator.
The OECD Indicators on the Governance of Sector Regulators form part of the work programme of the OECD Network of Economic Regulators and measure the governance of economic regulators in the energy, e-communications, rail transport, air transport and water sectors. The indicators cover regulators in all OECD countries and in many non-OECD countries. The Secretariat derives the indicators from a questionnaire, distributed alongside the OECD’s Product Market Regulation survey. In general, respondents to the questionnaire were high-level officials in regulatory agencies and/or relevant ministries. The responses undergo a rigorous data verification and validation process by the OECD Secretariat, verifying their completeness, consistency and accuracy in consultation with the respondents. The indicators are calculated by averaging equally weighted questions and sub-questions, to avoid imposing judgements about the importance of elements within the composite indicators. They are mapped on a scale from 0 (most effective governance arrangements) to 6 (least effective governance arrangements), in line with the Product Market Regulation methodology. The process of developing the questionnaire, collecting the data, validating responses, and analysing the results benefitted from the extensive support of the OECD Network of Economic Regulators. For a complete description of the methodology, see Casullo et al. (2019).
The indicators are divided into three components: independence, accountability and scope of action. The questions on performance fall within the accountability component, which covers the accountability of the regulator with regard to various stakeholders, including government, parliament, regulated industry and the general public.
Compliance with legal obligations refers to information on the regulator’s compliance with legal requirements, such as the fulfilment of information obligations or the proportion of decisions taken that are upheld.
Quality of the regulatory process refers to information on the performance of tools and processes used in decision making, such as impact assessment, stakeholder engagement and ex post evaluation.
Organisational/corporate governance performance refers to information on the internal functioning of the regulator, such as the timeliness of completion of planned activities, staff survey results and leadership performance information.
Operational service delivery refers to information on the delivery of the functions and responsibilities of the regulator, such as the number of inspections or provision of licences.
Further reading
Casullo, L., A. Durand and F. Cavassini (2019), “The 2018 indicators on the Governance of Sector Regulators – Part of the Product Market Regulation (PMR) Survey”, OECD Economics Department Working Papers, No. 1564, OECD Publishing, Paris, https://doi.org/10.1787/a0a28908-en.
OECD (2014), The Governance of Regulators, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/9789264209015-en.