Annex E. Methodology for composite indexes on Gender Budgeting, Regulatory Policy and Governance, and Open Useful Re-usable data

The narrowly defined composite indexes presented in Government at a Glance represent the best way of summarising discrete, qualitative information. “Composite indexes are much easier to interpret than trying to find a common trend in many separate indicators” (Nardo et al., 2005). However, their development and use can be controversial. These indexes are easily and often misinterpreted by users due to a lack of transparency as to how they are generated and the resulting difficulty to truly unpack what they are actually measuring.

The OECD has taken several steps to avoid or address common problems associated with composite indexes. The composites presented in this publication adhere to the steps identified in the Handbook on Constructing Composite Indicators (OECD/EU/JRC, 2008) that are necessary for the meaningful construction of composite or synthetic indexes.

Each composite index is based on a theoretical framework representing an agreed upon concept in the area it covers. The variables comprising the indexes are selected based on their relevance to the concept. Each index is constructed in close collaboration with the relevant OECD expert group including seeking their advice on the selection of the variables for the composite and the use of weighting schemes.

In addition, various statistical analyses are conducted to ensure validity and reliability of the composite indicators.

• The survey questions used to create the indexes are the same across countries, ensuring that the indexes are comparable.

• Different methods for imputing missing values have been explored.

• All sub-indicators and variables were normalised for comparability.

• To build the composites, all sub-indicators were aggregated using a linear method according to the accepted methodology.

• Principal component factor analysis is conducted to confirm hypotheses on the underlying concepts being measured.

• Redundant variables are excluded to avoid double counting and overweighting.

• Chronbach’s alpha is also calculated to measure inter-item correlations.

• Finally, sensitivity analysis (Monte Carlo simulation) is performed to establish the robustness of the indicators to different weighting options.

Detailed annexes on each of the composite indexes presented in this edition of Government at a Glance, including the variables and weights used to construct each indicator, are available online at http://www.oecd.org/gov/govataglance.htm.

References

Nardo, M., et al. (2005). Tools for Composite Indicators Building, Report EUR 21682 EN, Joint Research Centre, Ispra.

OECD/EU/JRC (2008), Handbook on Constructing Composite Indicators and User Guide OECD Publishing, Paris, https://doi.org/10.1787/9789264043466-en.

Annex E. Methodology for composite indexes on Gender Budgeting, Regulatory Policy and Governance, and Open Useful Re-usable data