Chapter 3. Assessment of coverage (Module 2)
This chapter provides guidelines to evaluate the adequacy and appropriateness of the existing social protection programmes in relation to risks and vulnerabilities identified in Module 1. It proposes a three-stage analysis, starting with an assessment of the institutional, political, and legislative context for social protection, followed by mapping the social protection system through a detailed inventory of current programmes; and finally overlaying existing provisions with the demand for social protection in order to identify coverage gaps.
Analytical dimensions
Module 2 catalogues existing social protection provision and assesses the extent to which it responds to a country’s present and future needs. It uses a three-stage methodology: analysing the institutional, political and legislative context for social protection; mapping existing programmes; and identifying gaps in the system relative to the drivers of demand for social protection identified in Module 1.
Analysing the legal, institutional and policy context for social protection indicates the extent to which the enabling environment exists for establishing a social protection system. Ideally, a country would have in place an overarching strategy for the sector that contains a firm policy commitment to establish a social protection system before embarking on the long-term project of doing so. A robust legislative framework for social protection – usually based in the individual’s rights enshrined in the Constitution – is also required to give effect to such a strategy.
Mapping various social protection instruments is often a complex task. In the absence of a well-defined system, social protection provision is typically implemented by many government institutions according to various legislative and policy imperatives, without taking into account possible gaps or overlaps among programmes. Schemes evolve at various points in time in response to various needs, with minimal co-ordination or information sharing. A government needs to understand the basic characteristics of all existing programmes that can or will form the basis for a new social protection system.
Once the mapping exercise is complete, the SPSR overlays existing provision with the demand for social protection identified in Module 1. In so doing, it identifies which groups are protected and which are not, and to what extent various risks are covered. With this information, a government can decide whether existing programmes need to be reformed or new programmes introduced (see Box 3.2 on understanding coverage within informality at individual and household levels).
Indicators and data sources
This module requires an in-depth desk review of legislative and strategic documents, which should be readily available. Consultations with policy makers and officials responsible for social protection design and implementation are crucial to ensure the full breadth of documents is included in the review and to understand discrepancies in implementation and legislative frameworks. Additionally, an assessment of data availability and gaps is carried out at this time to ensure that a full analysis can be conducted (Box 3.1). Overall, social protection performance indicators can be obtained from regional institutions, such as the Asian Development Bank, as well as global institutions, in particular the International Labour Organization (ILO) and the World Bank (Table 3.1).
Methodology
A first step in assessing the current state of social protection provision is to analyse social protection policies and strategies, as well as recent reforms and the legal basis for social protection. This high-level analysis will also include national statistics and data gathered by international organisations to provide an overview of the system’s performance and generate international comparisons with the benchmark countries identified in Module 1, for instance, on overall spending (Figure 3.1).
Sources: OECD (2018[1]), Social Protection System Review of Kyrgyzstan, https://doi.org/10.1787/9789264302273-en, based on ADB (2013[2]), Social Protection Index (database), hdl.handle.net/11540/79 (accessed December 2017).
The next step is a detailed inventory of social protection programmes and their characteristics for each pillar of the social protection system. This will include the basic information about programmes, such as their basis in law, type of transfer, eligibility criteria, coverage, and agency or institution responsible for implementation. The inventory can then be included as an annex in the format presented in Table 3.2.
It is important here to distinguish between the de jure and de facto coverage indicators whenever available: de jure coverage reflects the coverage established by virtue of laws, regulations or contracts, whereas de facto coverage reflects administrative practice. Discrepancies between these types of coverage can result from lack of implementation of laws or regulations, or inappropriate implementation, whether due to corruption, low take-up or other reasons.
Gathering data on various years is also of interest to analyse trends in the evolution of social protection programmes. For instance, in Kyrgyzstan, an analysis of the Monthly Benefit for Poor Families (MBPF) over time revealed a decrease in beneficiaries but an increase in benefit levels (Figure 3.2).
KGS = Kyrgyz Som.
Source: OECD (2018[1]), Social Protection System Review of Kyrgyzstan, https://doi.org/10.1787/9789264302273-en, based on NSC (2015[3]), Kyrgyz Integrated Household Survey (database), http://stat.kg/en (accessed June 2017).
The third step is to review the adequacy and appropriateness of existing social protection provision in relation to the previously identified risks and vulnerabilities. This analysis will reveal, for example, whether certain vulnerable groups are excluded from social protection, whether certain social protection schemes do not correspond to the country’s risk and vulnerability profile, or whether resource allocation within the social protection sector is optimal. This involves a cross-analysis with the vulnerable groups identified in Module 1.
Data availability and gaps are additional components covered in Module 2. In particular, the assessment should provide information on the identification mechanisms used at the operational level, for example, through single registries or social registries. The analysis should identify any gaps in the information system, in particular looking at various functions of intake and registration, assessment of needs and conditions, enrolment decisions, benefit levels or service package, and active case management (monitoring, grievance redress, etc.). It is key to assess the extent to which information is shared across agencies and ministries administering social protection programmes, which can be crucial in building or developing a social protection system.
Additionally, the SPSR often relies on microsimulations and descriptive statistics, based on household survey data. Unfortunately, household surveys may not include much information on social protection programmes (in particular, affiliation to, contributions or benefits), and the data may not be collected as frequently as necessary, thus offering an outdated and incomplete picture of the social protection system. It is important to circumvent these limitations, through two main channels:
-
1. Find other survey data that may provide information about social protection programmes. It is important, for example, to study labour force surveys or Demographic and Health Survey modules that may be relevant for the SPSR. In Cambodia, the SPSR team co-ordinated with the Ministry of Planning to access several waves of the IDPoor – the social registry database used to target and enrol beneficiaries in the Health Equity Fund – to analyse targeting accuracy and household transitions into and out of poverty. In Kyrgyzstan, the team complemented the Kyrgyzstan Integrated Household Survey with information from the Life in Kyrgyzstan panel.
-
2. Model missing information whenever possible, for example, by simulating a proxy means test or imputing data. In Indonesia, the Survei Sosial Ekonomi Nasional (SUSENAS), did not adequately capture household enrolment in the conditional cash transfer programme, Program Keluarga Harapan (PKH), underestimating coverage by about half. For this purpose, a probit regression was run using receipt of the grant as a dependent variable and a series of grant receipt determinants as predictors. The determinants, including household characteristics, receipt of other grants, and demographic and economic variables, were selected to maximise the regression’s explanatory power and goodness-of-fit. A probability threshold above which households are assumed to be PKH beneficiaries was then selected and was calibrated to reach the government-reported total beneficiary number. For additional robustness, the poverty rate (both regular and extreme or food poverty) among actual receiving households and those determined based on the probit were compared.
The Key Indicators of Informality based on Individuals and their Household (KIIbIH) database builds upon household surveys from 27 countries to provide comparable indicators and harmonised data on informal employment at individual and household levels across countries.
By focusing on both individuals and their households, and by covering a wide range of issues in the informal economy, such as employment, demographics, vulnerability and social protection, the database captures the heterogeneity of informal economy workers and takes into account their broader contexts, allowing comprehensive monitoring.
Unlike other publicly available harmonised statistics on informality, the KIIbIH is not based on labour force surveys. As such, it has a broader scope and provides a much wider information set related to workers’ households and socio-demographic and economic status. Consequently, it provides information on the degree of informality and enables classification of households as completely informal, completely formal or mixed. It thus allows monitoring of how workers’ vulnerability in the informal economy is transferred to other segments of the population and enriches the analysis and understanding of the various channels through which social protection can reach informal workers.
Overall, the database provides useful information for policy makers when designing and evaluating social protection systems. For instance, it facilitates estimating the number of individuals who may benefit from social insurance programmes as dependents in a household with at least one formally employed worker. This information can be further disaggregated to identify the number of children, working-age adults and/or elderly living in each type of household. Additionally, it provides detailed information on household consumption and income patterns, which serves as a useful basis to evaluate the contributory capacity of various types of households and to identify the profiles of workers who may be able to contribute, based on their location, household composition, and employment type and sector.
References
[2] ADB (2013), Social Protection Index, Asian Development Bank, Manila, http://hdl.handle.net/11540/79.
[3] NSC (2015), Kyrgyz Integrated Household Survey (database), National Statistics Committee of the Kyrgyz Republic, Bishkek, http://stat.kg/en (accessed on 01 June 2017).
[1] OECD (2018), Social Protection System Review of Kyrgyzstan, OECD Development Pathways, OECD Publishing, Paris, https://doi.org/10.1787/9789264302273-en.