3. Evidence for policy making in uncertain times
This chapter explores the ways in which real-time data, evidence gathering and rigorous experimental research help in the design of smart policies to meet the exceptional economic and health challenges of the COVID-19 crisis. It argues that development co-operation partners can support data-driven and active learning decision making in developing countries to help policy makers craft context-specific economic support and virus containment measures that protect lives and preserve livelihoods, as shown in Pakistan.
Developing countries have adopted economic support policies that reflect their limited fiscal capacity and their experience with social cash transfers.
Evidence is a priceless commodity and the lack of information available to policy makers and development co-operation partners means they struggle to fully understand how the pandemic, and strategies to contain it, impact livelihoods.
Development co-operation that supports the co-generation of research and data generation in real time, or as close to real time as possible, while also building local data capacity and systems can help developing countries take decisions based on data-responsive active learning.
The COVID-19 pandemic is an unprecedented crisis that will continue to impact the world for years to come. Many developing countries are enduring a triple economic shock: a decline in demand from trading partners; a decline in output due to containment measures; and the collapse of commodity prices. Policy makers face exceptional challenges due to the tremendous uncertainty over how best to mitigate its impact. Developing countries have varied in their policy response, both in terms of strategies to contain the virus and subsequent economic support packages to protect people.
At the International Growth Centre (IGC), our focus has been on improving rapid, real-time data collection to provide better evidence to policy makers in developing countries during these uncertain times. This is in line with our overall aim of working closely with policy makers and academics to co-generate research to support improved decision making. Part of the IGC’s COVID-19 response has been the development of an economic support policy tracker, which details the different levels and modes of fiscal support implemented by national governments to mitigate the economic downturn. Below we share the evidence from the tracker; propose a framework for how best to formulate containment strategies that incorporate active learning into the standard framework of decision making; and present a case study of active learning in Punjab, Pakistan, that demonstrates the promise of such an approach.
From the IGC economic support policy tracker, we find that most low- and lower middle-income countries have introduced or expanded some form of social support or cash transfers (Figure 3.1). This appears to be an attempt to mitigate against the sudden, sharp drop in incomes that puts millions of individuals at risk of falling into extreme poverty. The International Monetary Fund forecasts that GDP in sub-Saharan Africa will contract by 3.2% in 2020 (IMF, 2020[1]). This loss in economic activity will most likely translate into an increase in poverty rates. IGC research in Uganda, for instance, estimates that containment measures in Uganda may lead to an 8 percentage point increase in poverty (Younger et al., 2020[2]). The focus on cash transfers follows a pattern in recent years of developing countries centring social protection strategies on such regular, small amounts of income to help poor households smooth consumption and mitigate poverty (DFID, 2011[3]). A large body of evidence suggests consumption-based income support measures can support increases in consumption, food security and assets (Vaziralli, 2020[4]). However, analysis of social protection programmes in developing countries finds many are limited in coverage and often exclude the poorest due to poor data quality and the large presence of informality (Parekh and Bandiera, 2020[5]).
Policy makers may find it unfeasible to respond to the COVID-19 crisis with cash transfers alone, however, given their tight fiscal constraints, the vast scale of the pandemic and the transfer programmes’ lack of coverage of the most vulnerable. But there are lower cost policies that can complement the economic support response in developing countries, such as increasing capacity at customs outposts, facilitating mobile money payments or providing personal protective equipment to market vendors. At the same time, policy makers should minimise supply-side disruptions created by public health measures, such as having stronger international co-ordination of border-related policies, and ensuring curfews do not create wastage of perishable goods (Bouët and Laborde, 2020[6]).
The ICG tracker also shows limited use of tax relief measures and support to small and medium-sized enterprises in response to the pandemic in low-income countries. This is probably a reflection of two realities. The first is that low-income countries have less fiscal capacity than developed countries and so are unable to roll out similar support packages. The impact of COVID-19 has caused their fiscal space to shrink further. The government of Ghana estimates it will have a revenue shortfall of 2.1% of GDP while the Zambian government is estimating a 12% revenue shortfall for budgeted revenue in 2020 (Dzansi, 2020[7]; Ng’andu, 2020[8]). Projected budget deficits add to the pressure on the expenditure side, where countries needed to increase public health spending. The second reality is that countries of different income classifications have different economic structures. Most people in low-income countries work outside of the formal tax system, making it difficult to channel relief measures through the tax registry.
Note: The sample includes 29 low-income countries, 48 lower middle-income countries, 36 upper middle-income countries and 62 high-income countries.
Source: IGC (2020[9]), COVID-19 Policy Response Dashboard, https://www.theigc.org/covid-19/tracker.
In sum, the policy measures taken by developing countries largely reflect their economic structures, their limited fiscal capacity and prior experience with social cash transfer programmes. The presence of large informal sectors makes it extremely difficult to channel support through taxation or otherwise, to both the poorest and to small and medium-sized enterprises.
Initially, policy makers in both developing and developed countries had very limited information about the most appropriate strategy for containing the spread of the virus in their local contexts. Consequently, the response has varied across developing countries. At one extreme are developing countries that implemented measures similar to those in developed countries and imposed full national lockdowns relatively early in the pandemic, such as Rwanda and Uganda. At the other extreme are countries such as the United Republic of Tanzania, where containment measures were limited and public health messaging was conflicting (Resnick, Spencer and Siwale, 2020[10]).
IGC researchers recommend an approach of active learning when responding to uncertain crises, with the objective of achieving graded and data-responsive smart containment (Haas, Khan and Khwaja, 2020[11]). This approach involves building on an existing framework of decision making during times of uncertainty and also highlights a role for active learning.
The standard framework of decision making in the face of uncertainty comprises four components (Haas, Khan and Khwaja, 2020[11]):
decisions that will be the same regardless of the information obtained should be quickly taken
decisions that require additional information that is relatively costless should only be taken after this information has been collected
all decisions will have consequences that can and should be measured so they can generate new information.
Active learning involves the development of graded plans that are data-responsive. Figure 3.2 as outlined by Haas, Khan and Khwaja (2020[11]), demonstrates this approach.
Source: Authors
The key features of an operational action plan for a COVID-19 containment strategy include:
a critical role for data, both existing data as well as data collected throughout the process
clear policies that are still flexible and modular enough to be data-responsive
containment strategies where the intensity of measures is based on local conditions
community messaging and compassionate enforcement to ensure voluntary compliance
policies that consider both immediate considerations and longer term needs
testing, management and enforcement that can and should be managed by local authorities
openness to allow for partnerships with local entities to leverage existing capacity.
As responses to COVID-19 were sweeping across the world, concerns started to arise that developing country governments were too quick to mimic the response in developed countries, without the necessary budgetary resources to mitigate the economic fallout. This was no different in Pakistan, where a strict lockdown was introduced in late March 2020 that lasted several months. The IGC commissioned research in Pakistan to pilot a smart testing strategy in Punjab province that was designed by a multidisciplinary research team including IGC researchers and implemented with several departments in the government of Punjab. The aim was to examine how the government of Pakistan could deliver an evidence-driven, rapid policy response to maximise lives saved and minimise economic costs.
The testing strategy involved first splitting cities and districts into very small geographical units (of approximately 200 households). This geo-information was applied when sampling and testing through a combination of contact tracing and testing of frontline workers (e.g. medical staff, government officials, etc.), as well as pooled testing in the geographical units prioritised by the known prevalence of COVID-19. In all instances and given the scarcity of tests, emphasis was placed on administering tests in order of the magnitude of risk of exposure to COVID-19. Where tests were unavailable, phone surveys were conducted to try to identify the presence of symptoms. The information gathered from the strategy would allow COVID-19 response measures (e.g. quarantine, lockdown) to be implemented in more specific geographical areas. Throughout May 2020, the team implemented the smart sampling methodology in different stages and across various geographical units. As of early August, 645 geographical units in Lahore had been covered and 12 251 unique samples collected. The strategy itself did not extend past May.
The smart sampling strategy combined with pooled testing showed that a data-driven approach is feasible and cost-effective and could be carried out regularly and at scale. It enabled the effectiveness of various lockdown policies to be evaluated and, on top of the smart sampling, allowed researchers to conduct rapid and real-time data collection. If desired, this strategy could be modified to add data on health outcomes not related to COVID-19 as well as economic indicators. Such an approach would be beneficial not only for managing the disease – the immediate priority – but also the later process of economic recovery. A smart containment approach was subsequently adopted in Pakistan, though it was not necessarily tied to the researchers’ methodology. Nonetheless, the research helped build significant capacity in the government of Punjab’s health departments.
As part of the IGC’s COVID-19 response, we also commissioned a special call for research. We thought it was crucial to generate data and start experimentation in order to support decision making, both during the early uncertain stages of the pandemic as well as in preparation for the post-pandemic recovery period. One research strategy for bringing stronger evidence to policy making is the use of randomised control trials. Many of the studies commissioned as part of our special call were randomised control trials built on existing research in the field and therefore had baseline surveys and appropriate sampling frames. One of these is a study in Ghana that repurposed a cash transfer study to test the effects of cash transfers on compliance with social distancing and welfare. Another is a study in Uganda that is evaluating the resilience of relationships in informal contracts between different firms, as well as their employees. These types of rigorous studies will help policy makers and development co-operation partners understand the impacts of specific policies both in the short and medium term.
Data and evidence gathering play a critical role during times of uncertainty. The fluidity of the COVID-19 pandemic has required policy makers to be aware of developments in as close to real time as possible, something that is often missing in developing countries.
Experimentation is most useful if it results in active learning. In this respect, data and evidence gathering play a critical role during times of uncertainty. The fluidity of the COVID-19 pandemic has required policy makers to be aware of developments in as close to real time as possible, something that is often missing in developing countries. To support this, the IGC has implemented several high-frequency surveys to monitor the economic impact of COVID-19 across our country network, examples of which are presented in Box 3.1. To maximise the impact of these surveys, the data are made publicly available and will be presented with data visualisation techniques to better communicate them to policy makers and other stakeholders.
A phone survey in Sierra Leone measures indicators such as market prices, food security, hours worked and business closures and poses questions related to COVID-19 awareness and prevention. The data are provided in regular bulletins to policy makers, and many of the indicators are used directly in monitoring and evaluation frameworks within the Ministry of Finance’s Quick Action Economic Response Programme and the Ministry of Health and Sanitation’s Risk Communication and Social Mobilization group.
The IGC worked alongside Innovations for Poverty Action to collect data on the economic impact of the pandemic in developing countries through the COVID-19 Economic Impact Surveys joint initiative.
A survey of traders in Lagos, Nigeria found that domestic travel restrictions had a large impact on their ability to source inputs domestically, highlighting the need for policy makers to minimise supply-side bottlenecks (Bishi, Grossman and Startz, 2020[12]).
Developing country policy makers were dealing with significant challenges before COVID-19, wrestling with enduring problems such as inequality and economic growth that has not been inclusive. The crisis has amplified these challenges and added further uncertainty into policy making. International solidarity and resources are needed more than ever. To this end, and starting early in the crisis, the IGC has co-chaired an advisory group with the United Kingdom’s Foreign, Commonwealth and Development Office that discusses emerging policy issues in developing countries and feeds evidence into its response. Extremely limited early in the pandemic but beginning to quickly emerge, evidence has become an immensely useful commodity at a time when so much is unknown and crucial for a smart policy response. There are two key important lessons for development co-operation so far:
1. Support developing countries in designing smarter policies based on a framework of active learning: Developing countries’ reliance on cash transfers and impromptu containment strategies have produced mixed results. Given the magnitude of the crisis, the unprecedented levels of uncertainty and the limited resources at policy makers’ disposal, policy and programming need to be smarter. Designing smarter policy should rely on data and evidence generation alongside a framework of active learning. International development partners are well placed to support this approach by acting as a conduit for the sharing of expertise and evidence and assisting policy makers with policy design.
2. Focus on co-generating data and evidence alongside policy makers and researchers from developing countries: The pandemic underscores the importance of evidence and data in crafting an effective response to crises. The lack of information available to policy makers means they still struggle to fully understand the impacts of COVID-19 on livelihoods and how best to mitigate them. International development support to research organisations and policy makers to co-generate research and data can ensure high-quality results. Such research will have greater potential for application by bringing together global and local researchers and aligning with policy makers’ priorities.
Acting on these two lessons will not only support the COVID-19 response, but also bring benefits for the future. First, it will enhance our understanding and resilience against future crises, including pandemics. Allocating scarce fiscal resources in an efficient and effective manner requires making tough choices, and this is better done with more information. Second, co-generating research with local researchers has the potential to develop local capacity for the future. Last, building this infrastructure for data collection and use – with better capacity and relationships between policy makers and researchers – can potentially support other areas of policy making outside of crisis management, thus improving prospects for economic development and inclusive growth.
References
[6] Bouët, A. and D. Laborde (2020), “COVID-19 border policies create problems for African trade and economic pain for communities”, International Food Policy Research Institute blog, https://www.ifpri.org/blog/covid-19-border-policies-create-problems-african-trade-and-economic-pain-communities (accessed on 27 October 2020).
[3] DFID (2011), Cash Transfers Evidence Paper, United Kingdom Department for International Development, London, https://webarchive.nationalarchives.gov.uk/+/http:/www.dfid.gov.uk/Documents/publications1/cash-transfers-evidence-paper.pdf (accessed on 9 October 2020).
[7] Dzansi, J. (2020), “Ghana lifts the lockdown: Has the government reneged on its commitment to contain COVID-19 at all costs?”, International Growth Centre blog, https://www.theigc.org/blog/ghana-lifts-the-lockdown-has-the-government-reneged-on-its-commitment-to-contain-covid-19-at-all-costs (accessed on 9 October 2020).
[11] Haas, A., A. Khan and A. Khwaja (2020), “Policymaking in uncertain times: Smart containment with active learning”, Policy Brief, Center for International Development, Cambridge, MA and International Growth Centre, London, https://www.theigc.org/wp-content/uploads/2020/05/Haas-et-al-2020-brief_final.pdf (accessed on 9 October 2020).
[9] IGC (2020), COVID-19 Policy Response Dashboard, International Growth Centre, London, https://www.theigc.org/covid-19/tracker.
[1] IMF (2020), World Economic Outlook Update: A Crisis Like No Other, An Uncertain Recovery, International Monetary Fund, Washington, DC, https://www.imf.org/en/Publications/WEO/Issues/2020/06/24/WEOUpdateJune2020 (accessed on 9 October 2020).
[8] Ng’andu, B. (2020), 2021 Budget Address by the Honourable Dr. Bwalya K.E. Ng’andu, MP, Minister of Finance, Zambia National Assembly, Lusaka, http://www.parliament.gov.zm/sites/default/files/documents/articles/OFFICIAL-2021-BUDGET-SPEECH.pdf (accessed on 9 October 2020).
[5] Parekh, N. and O. Bandiera (2020), “Do social assistance programmes reach the poor? Micro-evidence from 123 countries”, Growth Brief, International Growth Centre, London, https://www.theigc.org/wp-content/uploads/2020/06/Parekh-and-Bandiera-2020-Growth-Brief.pdf (accessed on 9 October 2020).
[10] Resnick, D., E. Spencer and T. Siwale (2020), “Informal traders and COVID-19 in Africa: An opportunity to strengthen the social contract”, Policy Brief, International Food Policy Research Institute, Washington, DC and International Growth Centre, London, https://www.theigc.org/wp-content/uploads/2020/08/Resnick-et-al-2020-Policy-Brief.pdf (accessed on 9 October 2020).
[4] Vaziralli, S. (2020), “A social protection response to COVID-19 in developing countries”, Policy Brief, International Growth Centre, London, https://www.theigc.org/wp-content/uploads/2020/04/Vazirelli-2020-policy-brief.pdf (accessed on 9 October 2020).
[2] Younger, S. et al. (2020), “Estimating income losses and consequences of the COVID-19 crisis in Uganda”, paper presented at the 1st Session of the Monthly Virtual Peer-to-Peer Research Seminar Series, International Monetary Fund, Washington, DC.