Annex B. Methodology for analysing COVID-19 policy trackers
This annex describes the methodology used in Chapter 2 of Financing SMEs and entrepreneurs 2022: An OECD Scoreboard. The annex explains the objectives underlying the analysis and the different data sources and policy trackers used. It also describes in detail the methodology used and its limitations.
Chapter 2 aims to identify the SME orientation of policies in response to COVID-19, in order to assess how SME financing support is channelled through recovery packages. To this end, it distinguishes between “SME-related” and “other policies”; “SME-related” policies explicitly target SMEs or reference them as one of the target groups. “Other policies” do not mention SMEs specifically. The analysis looked both at the SME orientation by number and by value of policies.
This analysis of the SME orientation of policies was undertaken for both rescue and recovery measures. Following the definition in one of the databases used, rescue measures are defined as “short-term measures designed for emergency support to keep people and businesses alive”; recovery packages as “long-term measures to boost economic growth” (O’Callaghan, 2021[3]). Where possible, SME-related policies were differentiated by focus on firm age (e.g. start-ups), self-employed, type of entrepreneurs and firm size per se.
The analysis also assesses the SME orientation of policy measures at a lower level of aggregation. First, it provides a detailed analysis of the SME orientation of types of support (liquidity, alternative finance, insolvency) and financial instruments (debt, deferral, grants, factoring, leasing, equity) used. These types of financial support and instruments play a central role in rescue and recovery measures and also allow to position the analysis in the context of wider challenges and developments in SME finance, such as the importance of diversification of financial instruments for SMEs. Second, the analysis focuses on the SME orientation of measures in four key policy domains in the recovery packages: greening, digitalisation, skills and innovation.
Figure A B.1 shows the approach that was used.
Chapter 2 makes use of a number of policy tracking databases that have been developed since the start of the pandemic to monitor the policy response to the COVID-19 crisis. Some of these data sources have a specific focus (e.g. sustainable and green policies); some focus only on rescue or recovery measures, while others are more comprehensive in their scope, monitoring various types of policies that have emerged since the start of the COVID-19 crisis. The databases also vary in country coverage and in the level of detail of information they provide on policies.
The following databases were used:
Global Recovery Observatory (GRO): This database was developed by the Oxford University Economics Recovery Project (OUERP) and is continuously updated. It covers both rescue and recovery measures and includes information on the number and value of policies. Data used in Chapter 2 were last accessed in October 2021, and include 7 584 policies in 91 countries (O’Callaghan, 2020[4]).
Bruegel dataset EU Recovery and Resilience Facility: This dataset was developed by the Brussels-based think tank Bruegel and provides information on the Recovery and Resilience plans (RRF) of 22 EU countries, including 1 763 policies. The database was last accessed in July 2021 (Bruegel, 2021[5]).
The OECD Green Recovery Database: This database provides information on recovery plans that are likely to have significant environmental implications across 44 countries. 857 policies were included in the database as of September 2021 (OECD, 2021[6]).
Green Recovery Tracker: This dataset was developed by the Wuppertal Institute and E3G. It includes recovery measures in 17 EU member countries, and assesses them from the perspective of their expected impact on climate change. 996 policies were included in the database as of September 2021 (Wuppertal Institute, E3G, 2021[7]).
Covid-19 Government Financing Support Programme for Businesses – OECD: This dataset was developed by the Directorate for Financial and Enterprise Affairs (DAF) of the OECD to support the work of the OECD Committee on Financial Markets (CMF). Built on two waves of a survey among CMF Delegates (one in April 2020 with 26 responses and one in December 2020 with 21 responses), this database includes 215 financial support programmes for businesses (OECD, 2020[8]) (OECD, 2021[9])).
The analysis was conducted in three steps.
Step 1: Making use of relevant existing classifications in the databases
Each database includes a predefined set of categories or classes of policies, which have been used as a starting point for the analysis:
The analysis made use of the distinction between rescue and recovery measures included in the Global Recovery Observatory database.
To build the pool of SME-related policies, policies in Archetype C (“Liquidity for SMEs and start-ups”) in the Global Recovery Observatory database were included.
Furthermore, classifications on the type of policy objectives were used. This includes the use of the “Clean archetype” in the Global Recovery Observatory database and the “Green transition” and the “Digital transformation” classification in the Bruegel database.
Step 2: A structured text analysis
To further assess the SME orientation of policies in the databases, a structured text analysis was used based on a word search of relevant terms with respect to SME orientation, type of SME, financial instruments and types of financial support and policy domains. Descriptive texts on policies available in the databases were used to this end. Tables A.B.1, 2 and 3 show the search terms used.1
A set of dummy variables was built for each keyword (where 1 means that the description of the policy contains the term selected and 0 that this is not the case). The main objective was the creation of a "macro" dummy column which collects and takes into account the policies identified using pre-set categorisation and using the different keywords while avoiding the double counting of these policies that, by nature, overlap.
Step 3: Comprehensive manual check of results
To avoid any false positives, a manual check was performed on all policies in the different trackers. The quality check consisted of scanning and reading the policy descriptions where the keyword search was performed and testing if the methods applied produced accurate findings for each SME-related policy, policy domain and financial instrument, followed by a correction of the false positives. On average, 15% of the policies identified by type of SME were false positives. When looking by policy domain, 16% on average were false positives, while 14% in financial instruments and 15% by type of financial support. Furthermore, the manual check also focused on identifying financial outliers in the results, for instance because in some cases policies in the databases were presented as larger packages of measures instead of individual policies.
The methodology used allowed for an interpretation of the SME orientation of the policy response to COVID-19. However, the approach also has a number of limitations that needs to be taken into account when interpreting the results:
First, at the time when the databases were accessed, they were not always fully up to date, considering that they are continuously updated. As a consequence, not all policies put in place by countries could be included in the analysis. For instance, in October 2021 not all recovery packages in OECD countries had been included in the Global Recovery Observatory database.
Second, the various databases used differ in objectives, methodologies and country coverage, and are therefore not fully comparable.
Third, data on the values of policies in particular should be carefully considered. The policies in the databases mostly refer to the announcement of measures, not actual expenditure. In addition, not all policies have financial values attached to them, which can lead to an underestimation of the financial allocation. Moreover, values of different types of support in the databases were aggregated, but have different meanings, for instance the use of grants compared to loan guarantees. Last, changes in exchange rates may affect the comparability of policy values.
Finally, while the analysis provides relevant insights on the SME orientation of policies and their evolution, it is important to keep in mind that policies that are not “SME-related” may also be relevant for SMEs. Many policies aim to strengthen economic structures, such as broadband infrastructure, which benefits SMEs as well. Also, financial support measures open to the business sector at large can also be relevant for SMEs. Furthermore, the fact that the SME orientation of recovery policies is weaker than that of rescue policies is in part the logical consequence of a shift towards more generic policy measures. A normative interpretation of how high or low the share of SME-related policies should be is beyond the scope of this analysis.
References
[10] AIFI (2020), Il mercato italiano del private equity e del venture capital, Associazione Italiana del Private Equity, Venture Capital e Private Debt, https://www.aifi.it/visualizzaallegatodocumenti.aspx?chiave=u10jlmi71yQ3jShJ6O2Nz9nr8MV48g.
[1] British Business Bank (2021), Regions and Nations Trackers: Small Business Finance Markets, https://www.british-business-bank.co.uk/research/regions-and-nations-tracker-2021/.
[5] Bruegel (2021), European Union countries’ recovery and resilience plans, https://www.bruegel.org/publications/datasets/european-union-countries-recovery-and-resilience-plans/.
[2] Halabisky, D. and I. Basille (forthcoming), Policy brief on access to finance for inclusive and social entrepreneurship.
[11] NESTA (2009), The Vital 6 Per Cent: How High-Growth Innovative Businesses Generate Prosperity and Jobs, National Endowment for Science, Technology and the Arts, https://media.nesta.org.uk/documents/vital-six-per-cent.pdf.
[3] O’Callaghan, B. (2021), Global Recovery Observatory Methodology, https://doi.org/20210201-Global-Recovery-Observatory-Draft-Methodology-Document-.pdf (ox.ac.uk).
[4] O’Callaghan, B. (2020), Global Recovery Observatory, https://recovery.smithschool.ox.ac.uk/tracking/.
[9] OECD (2021), COVID-19 Government Financing Support Programmes for Business [DAF/CMF(2021)6/REV2], http://COVID-19 Government Financing Support Programmes for Businesses: 2021 Update (oecd.org).
[6] OECD (2021), Focus on green recovery, https://www.oecd.org/coronavirus/en/themes/green-recovery#climate-action.
[8] OECD (2020), COVID-19 Government Financing Support Programmes for Businesses, OECD, Paris, http://www.oecd.org/finance/COVID-19-Government-Financing-Support-Programmes-for-Businesses.pdf. (accessed on 28 February 2021).
[12] OECD (2015), New Approaches to SME and Entrepreneurship Financing: Broadening the Range of Instruments, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264240957-en.
[7] Wuppertal Institute, E3G (2021), Green Recovery Tracker, https://www.greenrecoverytracker.org/.
Note
← 1. Some of the terms used in the word search method do not display the full word, as the abbreviated form avoid different variations of the terms to not be considered. For instance, singular and plural terms (“deferral”-s), nouns and adjective or nouns and verbs (“environment”-al, “pollut”-ion and –ing). Considerable attention has also been given to the different ways words were mentioned in the databases, such as “AI” and “Artificial intelligence” and “start-up” and “startup” (without the dash). Moreover, significant efforts have been paid to avoid the misattribution of words (such as “small” and “connect”) to meanings unrelated to their context (that is, respectively, in this case, small businesses and digitalisation). Some of the terms contain the whole word to avoid potential confusion with similar others (as for “lease” and “leasing” that, if abbreviated, would have been confused with expressions such as “at -leas-t”) and they might also contain additional spaces to distinguish them from other words (“ lease ” and “ leasing ”, not to be included in terms such as “re-lease” and “re-leasing”).