9. What issues did COVID-19 SME and entrepreneurship support highlight for evaluation?
This chapter turns to the specific issue of assessing the impacts of government emergency support measures for SMEs and entrepreneurship during the COVID-19 crisis and learning the lessons for future crisis support. It begins by discussing the impact of the COVID-19 crisis on SMEs and entrepreneurship and highlighting the SME and entrepreneurship support measures that were introduced, covering labour support measures, deferrals of payments to governments, financial instruments and structural policies. The chapter then identifies three evaluation lessons for government crisis measures highlighted by the COVID-19 policy response – the need to specify objectives, identify expenditures, and evaluate impacts. It also makes a call for internationally co-ordinated evaluation of the crisis response.
This chapter examines the role of SME and entrepreneurship policy evaluation in guiding government policy responses to economic crises. There have been at least two global crises in recent decades that have required an important government SME and entrepreneurship policy response, notably the Global Financial Crisis of 2008-10, and the COVID-19 crisis, which started in 2020 and is still ongoing. More recently, measures have been introduced to support SMEs and entrepreneurs through the supply chain, trade and energy and commodity price disruptions of Russia’s large-scale aggression against Ukraine. Governments may also face the need to respond to more local shocks, including natural disasters. Governments need to be ready to introduce appropriate policy measures rapidly and effectively when such crises emerge.
The chapter makes the case that SME and entrepreneurship policy evaluation should play a central role in guiding government policy responses to major economic shocks. Evaluation has two crucial roles. The first is to help select policies that can be introduced quickly and successfully by drawing on knowledge of what has worked in the past. The second is to review, as they unfold, the relevance, effectiveness and efficiency of the policy responses to the shock, using evaluations of the new interventions introduced. This is needed both to adjust ongoing policies in the short term and to add to the evaluation evidence base for the future.
The chapter focuses on the example of the SME and entrepreneurship policy responses to the COVID-19-induced global economic shock. We argue that an effective evaluation of the medium- and long-term impacts of the COVID-19 support measures should be introduced in coming months. We also highlight weaknesses in terms of building in an evaluation component to the COVID-19 measures as they have been applied to date, which will need to be corrected for future crisis responses. The key issues involve setting clear objectives, identifying expenditures clearly and developing a programme to analyse appropriate data on SME and entrepreneurship activity.
There is strong merit in co-ordinating evaluation exercises and sharing information on evaluation results on government responses to crises at an international level. This could include an international exercise linking the broad scale and nature of the COVID-19 SME and entrepreneurship policy response in different countries to data on SME and entrepreneurship performance in these countries, including SME employment, sales, productivity and survival, and new firm entry.
Overall, the chapter argues that government crisis response measures, and hence the resilience of SME and entrepreneurship activity, and the economy as a whole, could be developed more quickly and cost-effectively by drawing on evaluation evidence from responses to previous shocks in their design and implementation. Evaluation should therefore be built into all government SME and entrepreneurship crisis response measures.
From early 2020, the COVID-19 crisis, and particularly the associated public health interventions such as social distancing (“lockdowns”), put the economy on hold and plunged the world into a deep recession. At the nadir of the crisis in Q2 2020, GDP across OECD countries was 11.6% lower than during the same period the previous year, although the gap reduced to 3.8% in the third quarter (OECD, 2021[1]).
While firms of all sizes were impacted – either directly or indirectly – SMEs were hit particularly hard given their overrepresentation in sectors that were most exposed to the containment measures (OECD, 2021[1]). Examples include the wholesale and retail trades, accommodation and food services, real estate, professional services, and other personal services (e.g. hairdressing).
The crisis triggered a major shock to the financial liquidity of SMEs and entrepreneurs. The survival of an SME depends heavily upon its short-term cash flow, because most lack the cash buffers on which larger firms can draw (Lundmark et al., 2020[2]). The, even temporary, disruption to cash flow caused by lockdowns disproportionately affects SMEs, compared with larger firms. Added to this were supply-chain disruptions, uncertainty for an extended period and a shift in demand from physical contact to e-commerce. These all presented significant challenges to SMEs and entrepreneurs.
COVID-19 not only threatened the survival of existing SMEs but also could be expected to influence business start-up decisions. Some people might decide to postpone or cancel their venture until circumstances change. This may well be a wise decision since recent studies have shown that, even in the long-run, new firms that begin in recessions under-perform those starting in more buoyant conditions ( (Sedláček and Sterk, 2017[3])). Others however might take the view that a lack of alternative income and employment opportunities means that starting their own business is the “least-bad” option open to them during difficult economic periods. Once again, the evidence on the impact of recessions on start-up rates is mixed but, after a thorough review, (Parker, 2018[4]) concludes that, when unemployment rises, so do rates of business creation – the recession-push effect. However, the “quality” of these start-ups is lower – in line with (Sedláček and Sterk, 2017[3]).
Available data indicate that SMEs were strongly affected by the crisis in terms of growth and liquidity, with 33-50% of surviving SMEs experiencing a fall in sales of more than 40% (OECD, 2021[1]). However, the major wave of bankruptcies that was initially feared did not come to pass (OECD, 2021[1]). At the same time, business creation rates were relatively robust in the face of the COVID-19 crisis. The number of firm entries rebounded after an initial drop upon the onset of the pandemic in 2020, although the pace and strength of this recovery varied across countries (OECD, 2021[5]). These outcomes are likely to result both from the adaptability of SMEs and entrepreneurs in transitioning to new operating practices, modes of working and markets, as well as the policy support measures introduced.
Governments were very active in seeking to support SMEs and start-ups from the beginning of the COVID-19 crisis, as of the first quarter of 2020. As set out in (OECD, 2020[6]), the responses frequently followed the following sequence:
Health measures and information to SMEs on how to adhere to them.
Measures to supply extra and more easily available credit to strengthen SME resilience.
Measures to avoid or mitigate the consequences of un-organised layoffs by extending possibilities for temporary redundancies and wage subsidies.
Structural policies to help SMEs fast track recovery and enhance long term resilience and growth.
Nevertheless, there were wide variations across countries in the scale, mix and timing of the measures introduced, reflected in part by the severity of the COVID-19 pandemic.
Governments typically pursued a mix of policy measures dedicated towards, or available for, SMEs and/or start-ups (OECD, 2021[7]). These include:
Labour support measures – wage support to compensate employees for reduced working hours and temporary redundancies, financial support to compensate for sick leave, and support to maintain the incomes of the self-employed.
Deferrals of payments to government and government-owned organisations – deferrals of income and corporate tax payments, value added tax, social security and pension payments, debt payments and rent, utility and local tax payments and accelerated payments of public procurement contracts.
Financial instruments – extended and simplified loan guarantees, direct lending through public institutions, grants and subsidies, and support for non-banking finance, e.g. equity funds. These measures, together with a surge in the demand for loans given the difficult economic conditions, meant that many countries saw an increase in SME lending in 2020 (OECD, 2022[8]).
Structural policies to strengthen SMEs and entrepreneurship in the recovery – advice and support to find new and alternative markets, introduce teleworking and digital technologies, innovate in new products and services, train and redeploy workforces, support start-ups, and modify insolvency and bankruptcy regimes to give firms breathing space.
Table 9.1 provides an overview of the measures that had been introduced by July 2020. They are placed in four groupings: Labour-related; Deferrals; Financial instruments and Structural interventions. Although there was some diversity between countries, wage subsidies were widely provided, as were tax deferrals. Finally, we see that direct lending to SMEs was widespread with only nine countries having no such programmes.
This re-emphasises the importance that SMEs and entrepreneurship play in economies and the role that they are expected to play in the recovery from COVID-19. It is important to note that the emphasis is placed on what we refer to elsewhere as “Hard” support; in contrast there is limited “Soft” support in the form of advice and training.
Initially, COVID-19 policies and programmes placed a strong focus on supporting SMEs to weather and survive the crisis, with relatively little emphasis on supporting start-ups. Indeed, only 10% of SME-focused policies in COVID-19 rescue packages were targeted at start-ups or entrepreneurs (OECD, 2022[9]). However, in later measures (recovery packages) somewhat greater attention was placed on structural reform, including support for business start-up (OECD, 2021[7]). Furthermore, certain groups often fell through the cracks of the emergency support, such as the self-employed with irregular or low incomes, recent start-ups and women entrepreneurs (OECD/European Commission, 2021[10]).
The total scale of support to SMEs and entrepreneurs appears to have varied strongly by country and there were also differences in the likelihood of different firms receiving support. In particular, younger and smaller SMEs were less likely to receive government support (OECD, 2023, forthcoming[11]). Governments are expected to gradually phase out these COVID-19 support packages.
It is critical to evaluate the impacts of the COVID-19 policy responses on SMEs and entrepreneurship to establish what worked and what did not work, effectively and efficiently, in achieving government objectives. However, applying the earlier guidance of this Framework to the case of the COVID-19 interventions leads us to identify three areas where the policies could have been better evaluated. These can be taken as three lessons for ongoing evaluation of the COVID-19 SME and entrepreneurship response measures and for the implementation of any future SME and entrepreneurship crisis responses.
Lesson 1: Objectives and Targets were often not clearly specified for the COVID-19 policy responses
A core message from our review of SME and entrepreneurship policy evaluations is that the Objectives and Targets of policy are often not clearly specified, making it difficult to evaluate policy effectiveness. This problem also appears to affect the SME and entrepreneurship policy measures rapidly introduced to deal with COVID-19, although it does not need to be the case for the medium- and longer-term measures aimed at strengthening the SME sector and promoting successful entrepreneurship in ongoing recovery packages.
Box 9.1 below suggests a range of possible Objectives but, because these are frequently in conflict with each other, it is important to clarify which take precedence and under what circumstances.
The aim of reducing unemployment in the short run (Objective 1) is a powerful influence on COVID-19 policy, but seeking to do this by saving all existing businesses (Objective 2) is likely to be expensive and also perhaps even undesirable on the grounds that many of these businesses would have exited without COVID-19. The currently favoured alternative – (Objective 3) – appears to be to save “viable” existing businesses, with such businesses being selected based upon the sector in which they trade.
The evaluation evidence in this report points to sector being only one of many influences upon the performance of an SME, implying that a sector-based approach is likely to be a crude tool for allocating funds. A clear alternative strategy is to focus less on the saving of SMEs and instead to focus on replacing those that have exited with new, and hopefully more dynamic, enterprises (Objective 4). The problem here is that, as noted above (Sedláček and Sterk, 2017[3]), new firms that begin in a recession have a relatively poor longer-run performance.
For this reason, policymakers might seek to be more selective about the firms they choose to support, by focusing on “better businesses” (Objective 5). Assuming that “better businesses” means those that are more likely to survive and prosper in a post-COVID-19 economy, the evidence from the evaluations reported here suggests that identifying such firms – particularly when the firm is new – is very difficult. These difficulties multiply when the key characteristics of the post-COVID-19 economy which they are entering are unclear1. However, as with Objective 3, governments should be making selection decisions based upon evaluation evidence.
A final, and radical, policy option is to acknowledge that recessions exercise a major “cleansing” role by eliminating businesses that become unprofitable once the recession is over (Objective 6). An example from COVID-19 might be the rise of on-line shopping which, if permanent, could lead to the closure of shops but an increase in transport-related suppliers.
The six Objectives set out above clearly imply very different policy responses. Each has both advantages and disadvantages and so imply the need to make political choices. For this reason alone, it is vital to be clear about – an ideally provide a rationale for – the Objectives of policy.
These Objectives are however incomplete unless also accompanied by a clear Target and a clear timescale for their completion. Examples of possible Targets and timescales are provided in Box 9.2. It draws a distinction between short- and long-run objectives.
In the short-run – say 12 months – only Objective 1 from Box 9.2 could realistically be achieved, albeit at considerable cost, using public funds to support the continued survival of all businesses.
If policy has a longer timescale, then it may be possible to replace much of the business stock within two years. However, this risks public funds being used to support both the start-up of new businesses likely to have only a very short life-span as well as the provision of funding to the “living dead” – i.e. businesses that survive only by being in receipt of public funds.
This might be addressed if policy were to have a longer time horizon. This would enable policymakers to be more selective about those start-ups and existing enterprises that policy supports. However, the policy would continue to encounter the problems of selection noted earlier.
The final, five-year, timescale could overcome many of the above problems, but is not likely to be favoured by politicians faced by an electoral cycle of less than five years.
It is important to clarify which Objectives take precedence and which Targets and timeframes are being aimed at.
So, what is policy success? Box 9.3 sets out some criteria that might be used by evaluators to reach a judgement on whether or not COVID-19 policy has “worked”, and which can be used by governments in choosing the policy mix which they see as likely to achieve these success metrics.
In summary, unquestionably, the core lesson from evaluation studies is that policy requires, as a minimum, for its Objectives and Targets to be clearly specified.
Lesson 2: Expenditure on COVID-19 SME and entrepreneurship policy responses is frequently difficult to identify
Information on COVID-19 policy response expenditures, both in aggregate and for certain specific programmes, can be difficult to obtain, but this is clearly a pre-requisite for assessing the success of the policy.
Table 9.2 gives an indication of the intensity of policy measures used by different countries in responding to COVID-19. It distinguishes between immediate fiscal measures, which include for instance grants and subsidies to SMEs (but also additional health expenditure), deferral of tax, social security and debt payments (for businesses and consumers) and other liquidity and guarantee instruments.
Currently this aggregate information is available only for a small number of countries. This may be because the data are available, but not in the public domain, or because they are not collected in a consistent format.
One of the key messages of this Framework is that no adequate assessment of policy impact is possible without the costs of that policy being documented. Given our focus on SMEs and entrepreneurship, it is important to know what proportion of the total COVID-19 response budget is spent on SME and entrepreneurship-targeted measures. This would be the first step in assessing whether, for example, public crisis-response funding is more cost-effective by supporting this group, rather than large enterprises.
Lesson 3: International evaluation evidence is important
The impact of government response measures to the COVID-19 pandemic, if only because the sums of money are so substantial, merits detailed investigation. Given the variety of policy packages and measures that were implemented across different countries, there is much to be gained from international comparative information on their impacts. This section provides the framework for such an investigation. To implement it requires the commitment of public policymakers and a range of challenges to be addressed by those undertaking the work.
First, individual programme evaluations need to be undertaken in each country using the rigorous evaluation techniques discussed in Parts I and II of this Framework document. The evidence from these evaluations should be drawn together across different countries and different types of interventions to provide comparative information on impacts and successes and failures, for example in a single data source or through a single meta-evaluation report. This will help policymakers derive lessons for responses to any future major shocks to SMEs and entrepreneurship.
Second, a broader evaluation exercise needs to relate the scale and nature of the COVID-19 SME and entrepreneurship policy response to the impacts achieved, recognising different contexts and objectives. Such a broad evaluation exercise would be expected to address the following key question:
Is there evidence that some types of SME and entrepreneurship policies were more effective than others in restoring economies to post-COVID-19 health?
For example, did the countries which provided loan guarantees – to existing SMEs – emerge more quickly/slowly than those using funding for the creation of new firms?
The challenges facing the analysts are to assess the impact of different COVID-19 policy measures for SMEs and entrepreneurship as operated in different countries, disentangling from differences across countries in the scale of the COVID-19 crisis, in other dimensions of government policy, and in the nature of the SME economy (by sector, company size, age, pre-existing financial health etc.). This would draw upon the techniques and approaches discussed in Parts I and II of this document.
Cross-section and time-series data – collected at country and ideally regional level – is required. This would document the policies and expenditures on SMEs and entrepreneurship and then link them to the economic outcomes on SMEs and entrepreneurship, accounting for the influence of the size of the COVID-19 shock. A final challenge is to specify – where this is not explicitly documented – the metrics for judging policy success.
Recommendation: Internationally co-ordinated policy evaluation should be undertaken on the impact of COVID-19 SME and entrepreneurship policy responses.
An important role could be played by the OECD Committee on SMEs and Entrepreneurship in undertaking an international review of the medium and long-term impacts of COVID-19 SME and entrepreneurship policy responses. This could involve assembling and interpreting information from individual evaluations of COVID-19 SME and entrepreneurship policy responses and a broader international analysis of the impact of different types of COVID-19 policy interventions on different SME and entrepreneurship performance metrics.
References
[12] Bruegel (2020), The fiscal response to the economic fallout from the coronavirus, https://www.bruegel.org/dataset/fiscal-response-economic-fallout-coronavirus.
[2] Lundmark, E. et al. (2020), “The Liability of Volatility and How it Changes Over Time Among New Ventures”, Entrepreneurship: Theory and Practice, Vol. 44/5, https://doi.org/10.1177/1042258719867564.
[8] OECD (2022), Financing SMEs and Entrepreneurs 2022: An OECD Scoreboard.
[9] OECD (2022), Financing SMEs and Entrepreneurs: An OECD Scoreboard.
[5] OECD (2021), Business dynamism during the COVID-19pandemic: Which policies for an inclusive recovery?.
[1] OECD (2021), OECD SME and Entrepreneurship Outlook 2021, OECD Publishing, Paris, https://doi.org/10.1787/97a5bbfe-en.
[7] OECD (2021), One year of SME and entrepreneurship policy responses to COVID-19: Lessons learned to “build back better”.
[13] OECD (2020), Coronavirus (Covid-19): SME Policy Responses.
[14] OECD (2020), E-commerce in the times of COVID-19, https://www.oecd.org/coronavirus/policy-responses/e-commerce-in-the-time-of-covid-19-3a2b78e8/#boxsection-d1e27.
[6] OECD (2020), Policy responses towards SMEs in the context of the COVID-19 virus outbreak.
[11] OECD (2023, forthcoming), OECD SME and Entrepreneurship Outlook 2023.
[10] OECD/European Commission (2021), The Missing Entrepreneurs 2021: Policies for Inclusive Entrepreneurship and Self-Employment, OECD Publishing, Paris, https://doi.org/10.1787/71b7a9bb-en.
[4] Parker, S. (2018), The Economics of Entrepreneurship, Cambridge University Press.
[3] Sedláček, P. and V. Sterk (2017), “The growth potential of startups over the business cycle”, American Economic Review, Vol. 107/10, https://doi.org/10.1257/aer.20141280.
Notes
← 1. For example, it is currently unclear the extent to which the increase in on-line purchases that occurred during the COVID-19 restrictions will decline, increase, or increase substantially in the longer term. The outcome has implications for retailing outlets, large and small, for hospitality and for transport.
← 2. See for the methodology: https://www.bruegel.org/publications/datasets/covid-national-dataset/