Annex A. Methodology for exploring fragility and projecting progress

There is broad consensus that fragility is dynamic and multifaceted: not all fragile states and economies face the same sets of social and institutional challenges, and they vary dramatically in the speed with which they build resilience over time. These aspects of fragility are difficult to capture in lists that present only a single snapshot in time or use a one-dimensional scale for fragility (UNDP/DIE, 2009). Accordingly, the first section of this annex presents several approaches to capturing the complexity of fragility, including:

A historical analysis of fragility, which disaggregates states and economies covered by the OECD monitoring list, to illustrate their varying trends and trajectories in fragility over time.
A presentation of five new fragility“lenses”, each focused on a goal endorsed by the post-2015 Open Working Group. The lenses illustrate the different forms of fragility that affect all states and economies to varying degrees, and identify groups of states and economies facing challenges that are generally not captured by the traditional monitoring list.
Alternative scenarios of rates of progress in addressing fragility. These projections illustrate the potential amount of time necessary for countries to reach a moderate level of success against the goal areas emerging from the Open Working Group (OWG).

This annex describes the methodology underlying each approach, and analyses trends and coding over time. It also presents the challenges in the proposed Sustainable Development Goals for the post-2015 era that can be addressed by fresh approaches to dynamic states of fragility.

Question 1: A historical perspective on the fragile states and economies list

The methodology the OECD uses to create the monitoring list has changed over time, limiting the degree to which lists from different years can be compared against one another, and the extent to which a given country’s trajectory over time can be assessed.1

In order to generate a list that can be compared across time, the authors replicated the OECD’s current methodology for producing its list, by combining the Fragile States Index and the World Bank fragile states lists for each year, from 2007 to 2015. Situations were only included in the composite list if they were identified as fragile in at least one year.

Question 2: A post-2015 perspective on fragility

Question 2 sets out a new approach to assessing aspects of fragility of particular relevance to the post-2015 agenda. The main goals of this approach are to identify groups of countries – both fragile and non-fragile – that will face similar challenges in making progress on the emerging post-2015 development agenda; to highlight countries facing stress factors that are likely to affect their ability to deliver sustainable development in the coming decades; and to illustrate differences in the geography and context of fragility, as compared to the traditional list of fragile states and economies.

The specific approach taken is to disaggregate fragility, by presenting five discrete indices and sets of country rankings. These indices are designed to measure goals drawn directly from the OWG’s proposed Goal 16. These include peaceful societies, justice for all, and effective and accountable institutions. Additional dimensions cover the economic fundamentals for peaceful and sustainable development, and resilience to shocks and hazards. For each goal proposed by the OWG, the authors designed a separate index, ranked all states and economies for which data were available (2012 data unless otherwise stated), and identified the 50 most vulnerable ones. For each goal proposed by the OWG, the authors designed a separate index. Each index is based on three datasets for each cluster (Table A.2 summarises the indicators and goal areas linked to each ranking). Each dataset was then rescaled to a scale from 0 to 100. The index is the average of these three numbers. Using this index, country lists were sorted from worst to best performing, and the 50 lowest performing countries and economies for each index are considered “vulnerable” or “challenged” in that area. Of particular interest in the context of this report were countries vulnerable across multiple dimensions.

Table A.2. Indicators of fragility based on the post-2015 Open Working Group goals
Fragility “lens”	Data sources for bottom-50 list Average of normalised indicators (scaled 0-100)	Relevant OWG goals
Peaceful and inclusive societies	Battle-related deaths, best estimate (BdBest) (Uppsala University, 2014) Interpersonal injuries, disability adjusted life years (DALYs) (WHO, 2014) Political instability - average of figures, perceptions of the likelihood of political instability and/or politically motivated violence including terrorism (World Bank, 2014b)	“Significantly reduce all forms of violence and related death rates everywhere.” (16.1)
Access to justice	Birth registration – percentage of children under age 5 whose births are registered, data available for countries between 2001 and 2012 (UNICEF, 2014) Control of corruption – average of figures, perceptions of the extent to which public power is exercised for private gain, Worldwide Governance Indicators (World Bank, 2014a) Rule of law – average of figures, perceptions of the extent to which agents have confidence in and abide by the rules of society (ibid.)	“Promote the rule of law at the national and international levels, and ensure equal access to justice for all.” (16.3) “Substantially reduce corruption and bribery in all its forms.” (16.5)
Effective and accountable institutions	Government effectiveness – average of figures, perceptions of public services, quality of civil service, its degree of independence from political pressures, quality of policy formulation, credibility of government’s commitment to policies (ibid.) Regulatory quality – average of figures, perceptions of the ability of government to formulate and implement sound policies and regulations that permit and promote private-sector development (ibid.) Voice and accountability – average of figures, perceptions of the extent to which a country’s citizens are able to participate in selecting their government, as well as freedom of expression, association and a free media (ibid.)	“Develop effective, accountable, and transparent institutions at all levels. (16.6)” “Ensure public access to information and protect fundamental freedoms, in accordance with national legislation and international agreements.” (16.10)
Economic foundations	Doing Business Index – ease of doing business, average of a country’s percentile rankings on ten topics (World Bank, 2014b) Education years – scaled years of education (Barro and Lee, 2010) GDP growth – ten-year average (IMF, 2014)	“By 2020 substantially reduce the proportion of youth not in employment, education, or training.” (8.6)
Resilience	Income inequality (Gini Index) – International Futures model (IFs, 2014) Healthcare capabilities (US National Intelligence Council, 2008) Vulnerability – estimates the disaster risk of a country using internal and external factors, 2014, World Risk Index (UNU-EHS, 2013)	“By 2030 build the resilience of the poor and those in vulnerable situations, and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters.” (1.5) “Strengthen resilience and adaptive capacity to climate related hazards and natural disasters in all countries.” (13.1)

Box A.1. Reducing fragility: Selected targets proposed by the Open Working Group on Sustainable Development Goals

Promote peaceful and inclusive societies

Significantly reduce all forms of violence and related death rates everywhere (16.1).
End abuse, exploitation, trafficking and all forms of violence and torture against children (16.2).
Strengthen relevant national institutions, including through international co-operation … for preventing violence and combating terrorism and crime (16.a).
Eliminate all forms of violence against all women and girls in public and private spheres, including trafficking and sexual and other types of exploitation (5.2).
Eliminate all harmful practices, such as child, early and forced marriage and female genital mutilations (5.3).
[…] provide safe, non-violent, inclusive and effective learning environments for all (4.a).
[…] by 2025 end … recruitment and use of child soldiers (8.7).
Strengthen prevention and treatment of substance abuse, including narcotic drug abuse and harmful use of alcohol (3.5).

Justice for all

Promote the rule of law at the national and international levels, and ensure equal access to justice for all (16.3).
By 2030 significantly reduce illicit financial and arms flows, strengthen recovery and return of stolen assets, and combat all forms of organised crime (16.4).
Substantially reduce corruption and bribery in all its forms (16.5).

Effective, accountable and inclusive institutions

Develop effective, accountable and transparent institutions at all levels (16.6).
Ensure responsive, inclusive, participatory and representative decision making at all levels (16.7).
Broaden and strengthen the participation of developing countries in the institutions of global governance (16.8).
By 2030 provide legal identity for all including birth registration (16.9).
Ensure public access to information and protect fundamental freedoms, in accordance with national legislation and international agreements (16.10).
Promote and enforce non-discriminatory laws and policies for sustainable development (16.b).

Economic foundations

By 2020 substantially reduce the proportion of youth not in employment, education or training (8.6).
By 2030 empower and promote the social, economic and political inclusion of all […] (10.2).
Facilitate orderly, safe, regular and responsible migration and mobility of people […] (10.7).
By 2030 ensure universal access to affordable, reliable and modern energy services (7.1).
By 2030, ensure access for all to adequate, safe and affordable housing and basic services, and upgrade slums (11.1).

Increasing resilience

By 2030 build the resilience of the poor and those in vulnerable situations, and reduce their exposure and vulnerability to climate-related extreme events and other economic, social and environmental shocks and disasters (1.5).
By 2030 significantly reduce the number of deaths and the number of affected people and decrease by y% the economic losses relative to GDP caused by disasters […] (11.5).
Strengthen resilience and adaptive capacity to climate-related hazards and natural disasters in all countries (13.1).
[…] strive to achieve a land-degradation neutral world (15.3).

Source: Open Working Group (2014), “Introduction to the Proposal of the Open Working Group for Sustainable Development Goals”, received 21 July 2014.

Question 3: Scenarios

Question 3 explores the ambition of the post-2015 agenda, particularly for fragile states and countries facing challenges in several areas likely to be emphasised in global goals and targets. The core analytical tool is a series of projections that estimate how long a given country or economy will take to reach a target level of capacity under differing assumptions. These projections are not intended as predictions, but rather as ways to benchmark and examine expectations regarding rates of progress on development objectives.

Projections were developed for a core goal area for which cross-national time series data were available. The projections were designed in three stages: goal setting, estimating rates of change and computing a set of potential “goal achievement years” for each country under different assumptions about rates of progress. This approach explicitly draws on methodologies developed by Pritchett et al. (2013).2

The indicators employed are the same as in the Institutions Cluster (see previous section concerning Question 2): regulatory quality, voice and accountability, and government effectiveness, all from the World Bank’s World Governance Indicators.

Goal setting

For the Institutions Index, the 70th percentile value was chosen as the target objective. This goal is arbitrary, but was chosen as a threshold that is both ambitious and realistic. Countries were ranked by their 2012 values, from best to worst, and the 70th percentile value was identified, by using the lowest value and the highest value to scale the scores from 0 to 100.

The Institutions Index goal is 0.02; sample countries around the Institutions goal value include Tuvalu (0.019) and Suriname (-0.002). In short, the goal for the Institutions Index is to reach Suriname’s approximate level of development.

Estimating rates of change

The authors then identified historical rates of institutional change for each country, by computing the average point per year change in the indicators for the Institutions Index, using all available data points from 1996-2012. Estimates were applied to the disaggregated data and then averaged. This process was used to derive three scenarios for each country.

The business-as-usual projection takes the average historical rate of change and assumes that it will continue into the future. For instance, if a given country historically improved its “voice and accountability” by 0.02 points per year, this average is projected forward in time.

The modestly optimistic projection was developed by identifying the 70th percentile rate of improvement for each indicator. Countries were ordered by rate of change from 1996 to 2012, from best to worst, and the 70th percentile rate was found (using the lowest value and the highest value to scale the scores from 0 to 100). The 70th percentile rates for each indicator are:

Voice and accountability: 0.044 points per year.
Government effectiveness: 0.0232 points per year.
Regulatory quality: 0.043 points per year.

The best-case scenario was developed by identifying the average rate of change for the top three countries and economies that had the fastest historical rate of improvement for each indicator. The top three countries and economies and average rate for each sub-indicator are:

Voice and accountability: 0.093 points per year (Kosovo, Niger, Serbia).
Government effectiveness: 0.067 points per year (Afghanistan, Georgia, Rwanda).
Regulatory quality: 0.085 points per year (Georgia, Rwanda, West Bank and Gaza Strip).

Identifying goal achievement year

A range of goal completion years was estimated for each country. The estimates were derived by taking the rate of improvement under each scenario and projecting it forward from the most recent available year of data, which is 2012. For each year, indicator values were averaged into the Institutions Index, and compared to the goal values (given above). For each scenario, the first instance in which a country’s score matches or exceeds the goal value is recorded as the date the goal is reached. Countries that did not reach the target by 2063 are marked as “after 2063”. Countries with negative baseline values – worsening rather than improving scores – are coded as never reaching the target.

Question 4: Aid metrics

Official development assistance excluding debt relief

Unlike previous years’ Fragile States reports, this report uses official development assistance (ODA) excluding debt relief as its primary metric in examining aid flows. This choice allows the study of flows that enter developing countries, instead of debt that has been forgiven on a country’s balance sheet.

Other official flows excluding debt relief – A first look at non-concessional flows to fragile states

This is the first year that the Fragile States report examines non-concessional other official flows (OOF) to fragile states and economies. To correspond with the removal of debt relief from the ODA figures, debt relief was also deducted from the OOF data figures.

Converting financial flows from current USD

It was decided for this report to convert current USD figures into constant 2012 USD using the corresponding US GDP deflator to show financial flows over time in real terms. However, it should be noted that using this methodology to convert the flows into constant USD is intended to capture the scale of the amount of flows from the development assistance provider’s perspective and may or may not be an accurate representation of the scale of the amount from the recipient country’s perspective.

Question 5: Data availability – Financial flows to fragile states

In examining financial flows in aggregate, per capita and percentage of gross domestic product (GDP), data for these flows in one or more years were missing for many fragile states. As seen in Table A.3, in 2012, data for remittances, OOF excluding debt relief, foreign direct investment (FDI) and ODA excluding debt relief are incomplete.

Table A.3. Data availability for three main external financial flows to fragile states, 2012
Flow	% of countries with data
Foreign direct investment	94
Remittances	64
Official development assistance (excluding debt relief)	100
Other official flows (excluding debt relief)	80
Sources: FDI and remittances data from World Bank (2014a), “Foreign direct investment, net inflows (% of GDP)”, World Development Indicators (database), available at: http://data.worldbank.org/indicator/BX.KLT.DINV.WD.GD.ZS; FDI as a percentage of GDP converted to USD using World Bank (2014b), “GDP figures (in current USD)”, World Development Indicators (database), available at: http://data.worldbank.org/indicator/NY.GDP.MKTP.CD; World Bank (2014c), “Personal remittances, received (current USD)”, World Development Indicators (database), available at: http://data.worldbank.org/indicator/BX.TRF.PWKR.CD.DT; OECD (2014a), “Detailed aid statistics: ODA official development assistance: Disbursements”, OECD International Development Statistics (database), https://doi.org/10.1787/data-00069-en; OECD (2014b), “Detailed aid statistics: Other official flows OOF”, OECD International Development Statistics (database), https://doi.org/10.1787/data-00075-en.

Beyond the question of data gaps in financial flows data, there is a more fundamental issue of missing data in population and GDP estimates. As a consequence, some fragile states that have data on financial flows cannot be counted in per capita or percentage GDP calculations. This further complicates the task of finding per capita and percentage GDP figures that are truly representative of all countries and economies on the fragile states list.

As a result of all of these issues, the figures presented in this report should be taken as approximations using the best available data sources but may deviate, in some cases potentially significantly, from the actual true values of the financial flows to fragile states. Better data quality in this area would greatly help to improve confidence and resulting policy recommendations on financial flows to fragile states in the future.

Question 6: Matching aid to the New Deal Peacebuilding and Statebuilding Goals

In order to estimate the amount of aid being directed to areas related to the Peacebuilding and Statebuilding Goals (PSGs), this report examined aid allocated at the individual project level through the OECD Creditor Reporting System (CRS).3 A manual matching process was performed between the five PSG areas and the descriptions of the OECD’s sector codes in order to approximate the scale of the amount of aid directed at projects in line with each of the PSGs. As mentioned in Chapter 5, the resulting figures should be seen as a rough estimate. To obtain more precise numbers, it would be necessary to analyse the specifics of each of the thousands of individual projects listed in the CRS database and to evaluate systematically whether a particular project contributed towards one or more of the PSGs. To avoid double counting, projects were only counted toward one PSG, which in some cases was an artificial distinction as there is definite overlap between the five PSG areas. It was particularly difficult to differentiate between projects contributing to areas related to PSG 4 (economic foundations) and PSG 5 (revenues and services). In presenting the data, therefore, aid allocated to either area was combined into one lump sum (see Figure 3.7 [Q.5]). These limitations should all be taken into consideration when viewing and interpreting the relative amounts of donors’ aid allocations to the five PSG areas. If anything, doing this matching highlighted the need for a better tracking system for aid toward PSG-related areas.

References

Barro, R. and J.-W. Lee (2010), “A new data set of educational attainment in the world, 1950-2010”, Journal of Development Economics, Vol. 104, pp. 184-198, accessed at: www.barrolee.com.

Burt, A. et al. (2014), “Eradicating poverty in fragile states: Prospects of reaching the ‘high-hanging’ fruit by 2030”, Policy Research Working Papers, Vol. 1, No. 7002, World Bank, Washington, DC.

Carleton University (2012), “Failed and fragile states”, Country Indicators for Foreign Policy, Carleton University, Ottawa, Ontario, available at: www4.carleton.ca/cifp/ffs.htm (accessed 1 August 2014).

FFP (2014), “Fragile States Index 2014”, The Fund for Peace, Washington, DC, available at: http://ffp.statesindex.org.

International Futures (IFs) model (2014), Version 7.08, Frederick S. Pardee Center for International Futures, Josef Korbel School of International Studies, University of Denver, Denver, Colorado, www.ifs.du.edu.

IMF (2014), World Economic Outlook: Legacies, Clouds, Uncertainties, International Monetary Fund, Washington, DC, October.

OECD (2014a), “Detailed aid statistics: ODA official development assistance: Disbursements”, OECD International Development Statistics (database), https://doi.org/10.1787/data-00069-en.

OECD (2014b), “Detailed aid statistics: Other official flows OOF”, OECD International Development Statistics (database), https://doi.org/10.1787/data-00075-en.

OECD (2014c), Fragile States 2014: Domestic Revenue Mobilisation in Fragile States, OECD Publishing, Paris, available at: www.oecd.org/dac/governance-peace/conflictandfragility/docs/FSR-2014.pdf.

Open Working Group (2014), “Introduction to the Proposal of the Open Working Group for Sustainable Development Goals”, received 21 July 2014.

Pritchett, L. et al. (2013), “Looking like a state: Techniques of persistent failure in state capability for implementation”, Journal of Development Studies, Vol. 49, No. 1, Taylor & Francis, London, https://doi.org/10.1080/00220388.2012.709614.

Pritchett, L. and F. de Weijer (2010), “Fragile states: Stuck in a capability trap?”, World Development Report 2011: Background paper, available at: http://siteresources.worldbank.org/EXTWDR2011/Resources/6406082-1283882418764/WDR_Background_Paper_Pritchett.pdf.

Rice, S.E. and S. Patrick (2008), “Index of State Weakness in the Developing World”, The Brookings Institution, Washington, DC, available at: www.brookings.edu/~/media/research/files/reports/2008/2/weak%20states%20index/02_weak_states_index.pdf.

UNDP/DIE (German Development Institute) (2009), A User’s Guide to Measuring Fragility, United Nations Development Programme, available at: www.undp.org/content/undp/en/home/librarypage/democratic-governance/oslo_governance_centre/governance_assessments/a_users_guide_tofragility.html.

UNICEF (2014), “UNICEF data: Monitoring the situation of women and children”, available at: http://data.unicef.org/child-protection/birth-registration.

UNU-EHS (2014), World Risk Report 2014, United Nations University Institute for Environment and Human Security, Bonn, Germany, available at: www.ehs.unu.edu/article/read/world-risk-report-2014.

Uppsala University (2014), Dataset, v.5-2014, 1989-2013, Conflict Data Program, Uppsala University, Uppsala, Sweden, available at: www.pcr.uu.se/research/ucdp/datasets/ucdp_battle-related_deaths_dataset (accessed 1 August 2014).

US National Intelligence Council (2008), “Strategic implications of global health”, www.dni.gov/files/documents/Special%20Report_ICA%20Global%20Health%202008.pdf; National Center for Medical Intelligence, Health-Care Capabilities, Data are available in form of a map at: www.dni.gov/files/documents/Special%20Report_ICA%20Global%20Health%202008%20foldout.pdf.

World Bank (2014a), “Foreign direct investment, net inflows (% of GDP)”, World Development Indicators (database), available at: http://data.worldbank.org/indicator/BX.KLT.DINV.WD.GD.ZS.

World Bank (2014b), “GDP figures (in current USD)”, World Development Indicators (database), available at: http://data.worldbank.org/indicator/NY.GDP.MKTP.CD.

World Bank (2014c), “Personal remittances, received (current USD)”, World Development Indicators (database), available at: http://data.worldbank.org/indicator/BX.TRF.PWKR.CD.DT.

World Bank (2011), World Development Report 2011: Conflict, Security, and Development, World Bank, Washington, DC, available at: http://siteresources. worldbank.org/INTWDRS/Resources/WDR2011_Full_Text.pdf.

Notes

← 1. The 2007 list was assembled by identifying states in the bottom two quintiles of the World Bank’s Country Policy and Institutional Assessment (CPIA) ranking as fragile; it also included non-ranked states such as the Democratic People’s Republic of Korea as well as several states that clustered just above the fourth quintile cutoff. From 2008-10, the list was compiled using the bottom two CPIA quintiles, data from the Brookings Index of State Weakness in the Developing World (Rice and Patrick, 2008 and the Carleton University Country Indicators for Foreign Policy [CIFP] Index, 2007). Since 2010, the fragile states list has been produced by combining the Harmonized List of Fragile Situations produced by multilateral development banks including the World Bank, the Asian Development Bank and the African Development Bank, with countries scoring 90 or above on the Fragile Country States Index produced by The Fund for Peace.

← 2. See also World Bank (2011) and Pritchett and de Weijer (2010).

← 3. CRS data can be found at: http://stats.oecd.org.