Annex A. Composite indicator on Use of a medium-term perspective in the budget process

This edition of Government at a Glance: Latin America and the Caribbean 2020 includes a composite index on the use of a medium-term perspective in the budget process. Data used for the construction of the composite are derived from the 2018 OECD/IDB Survey on Budget Practices and Procedures. Survey respondents were predominantly senior officials in the Ministry of Finance.

The narrowly defined composite indexes presented in Government at a Glance: Latin America and the Caribbean 2020 represent the best way of summarising discrete, qualitative information on key aspects of budgetary practices such as medium-term expenditure frameworks and performance budgeting. “Composite indexes are much easier to interpret than trying to find a common trend in many separate indicators” (Nardo et al., 2004). However, their development and use can be controversial. These indexes are easily and often misinterpreted by users due to a lack of transparency as to how they are generated and the resulting difficulty to comprehend what they are actually measuring.

The OECD has taken several steps to avoid or address common problems associated with composite indexes. The composites presented in this publication adhere to the steps identified in the Handbook on Constructing Composite Indicators (Nardo et al., 2008) that are necessary for the meaningful construction of composite or synthetic indexes.

The composite index is based on a theoretical framework representing the concept of medium-term fiscal framework. The variables comprising the index were selected based on their relevance to the concept by a group of experts within the OECD and in consultation with country delegates to the relevant working parties:

• Various statistical tools, such as factor analysis, were employed to establish that the variables comprising the index are correlated and represent the same underlying concept.

• Different methods for imputing missing values have been explored.

• All sub-indicators and variables were normalised for comparability.

• To build the composite, all sub-indicators were aggregated using a linear method according to the accepted methodology.

• Sensitivity analysis using Monte Carlo simulations was carried out to establish the robustness of the indicators to different weighting options (e.g. equal weighting, factor weighting and expert weighting). Expert weighting resulted as the most appropriate weighting method.

The index does not purport to measure the overall quality of budgetary systems. To do so would require a much stronger conceptual foundation and normative assumptions. Rather, the composite index presented in Government at a Glance: Latin America and the Caribbean 2020 is descriptive in nature, and has been a label that reflects this. The survey questions used to create the index are the same across countries, ensuring that the results are comparable.

While the composite index was developed in co-operation with OECD countries and are applied to the LAC region, it is based on best practices and/or theory; both the variables comprising the composite and their weights are offered for debate and, consequently, may evolve over time. The OECD is currently redefining best practices for budget transparency and is revisiting the concept of budgetary flexibility; as such, no composites related to these topics are presented in this edition.

The composite was built according to the following methodology: the topic was divided into broad categories comprising the theoretically relevant aspects for a medium-term expenditure framework. A weight was assigned to each of these broad categories. Within each of the broad categories, the relevant questions were identified, a sub-weight was assigned to each question and a score was given to each of the answers within these questions. The country scoring for each question is the product of the weight of the broad category and the sub-weight of the question multiplied by the answer provided by each country (1 or 0). The composite is the result of adding together these scores for each country. Both composites vary from 0 to 1; a score of 1 implies the use of sound practices.

Use of a Medium-Term Expenditure Framework (MTEF) at the central level of government variables, weights and scoring

The following items and weights have been used in the construction of the MTEF

Figure A.1. Variables and weights used in MTEF index