7. What are the implications of the evaluation evidence for refocusing SME and entrepreneurship policy?
This chapter draws upon the evaluation evidence assembled in Part II of this Framework to provide guidance for policy makers on refocusing SME and entrepreneurship policy to where it can be most effective. It responds to the challenges described in chapter 6 by highlighting three policy choices. The first is the policy balance between “Hard” or “Soft” interventions. The second is the extent to which policy is selective – focusing on certain “types” of firms – or more wide ranging. The third is the extent to which the policy is a “Macro” or “Micro” intervention. Recommendations are made across these three themes.
As discussed previously in this report, the evaluation of SME and entrepreneurship policies remains incomplete and the findings, even from well-conducted studies, are mixed. More widespread and reliable evaluation evidence is needed to show where SME and entrepreneurship policy actions are the most effective and efficient. Nevertheless, in this less-than-perfect world we select, as illustrations, three important policy debates. These are the hard/soft debate, the selectivity debate, and the macro/micro debate. We offer our interpretation of how the findings from evaluations provide policymakers with insights that enable them to make better informed decisions in these areas.
Part II of this Framework drew a distinction between evaluation cases that related to “Hard” or “Soft” policies and programmes. “Hard” are defined as those where an actual or in-kind financial payment was made to an enterprise or individual – perhaps in the form of a grant, loan or tax relief. This contrasted with “Soft” policies, which comprised the provision of advice, coaching, mentoring and counselling or skills development and training.
In Part II, 33 programmes were defined as Hard, 11 were Soft and 6 were Both (mixing both Hard and Soft). Although the numbers were small, Soft programmes were more likely to be classified as having no/negative outcomes than the Hard programmes. Of the 6 evaluations with “no/negative” outcomes, 50% were Soft programmes, although they constituted only 22% of all programmes. The comparable figure for Hard programmes having no/negative outcomes was 2 out of 33 (6%). Finally, 1 out of the 6 mixed (both hard and soft) programmes had no/negative outcomes. The Hard programmes were more likely to have mixed impacts than the Soft programmes, although the share of positive outcomes amongst Hard and Soft programmes was equal in this small set of evaluations. The evidence is inconclusive, but points to the need for further comparative investigation.
In addition, there are some grounds for thinking that Soft programmes are less likely to be evaluated than Hard programmes, and that when they are examined, they are subject to less intense scrutiny. For example, Part I reported the findings of an international review of 66 Soft start-up support programmes for young people (Sara, 2016[1]). It found only one programme that clearly specified Objectives and Targets and used a reliable counter-factual evaluation. This lack of reliable evaluation may contribute to the fact that relatively few evaluations of Soft support programmes could be included in Part II, since only reliable evaluations are included.
It is not possible, on a limited set of diverse evaluations, to make any strong statement on the superiority of Hard over Soft programmes in promoting enterprise. Moreover, there is reliable evidence that, in some cases, they do “work” (Rotger, Gørtz and Storey, 2012[2]). Nevertheless, larger numbers of reliable evaluations are needed of Soft programmes to justify public expenditures when the possibility of no impact is present. This issue has continued to be a concern since it was observed in the original OECD evaluation framework (OECD, 2007[3]), pointing to an “unproven” verdict hanging over Soft programmes.
Recommendation: Governments should look carefully, using at least Step V methods, at the impact of their existing, and any new proposed, “Soft” programmes.
Part I reported that two findings persistently emerged from reviews of the performance of a cohort of new firms. The first is that short-run survival rates are low; the second is that, in the medium term, job creation is heavily concentrated in a small proportion of firms. The latter are popularly referred to as “High Potential Start-Ups”, “Gazelles”, “Winners” or “Exceptionally Productive Job Creators (EPJCs)”. The standard “rule of thumb” is that 4% of new firms which start, provide 50% of the jobs in the cohort at the end of a decade. Unsurprisingly this statistic is of considerable interest to policymakers on the grounds that SME and entrepreneurship policy would be more cost-effective (if job creation is the sole objective) if it were focussed on the 4%, rather than on all start-ups.
There has been a healthy debate on the extent to which SME and entrepreneurship policy should focus on “winners”. This includes exchanges on whether it is possible to identify the fast growers and on whether, once picked, they continue to grow as rapidly. The case in favour of focusing on winners is primarily based on avoiding public funds being used to finance businesses that clearly have low survival rates and little potential to grow (Acs et al., 2016[4]). The case against is based on the temporal volatility of new and small firms – so that fast growers in one period of time return to the industry average in the next period (Parker, Storey and van Witteloostuijn, 2010[5]), (Daunfeldt and Halvarsson, 2015[6]). Therefore, even if they wished to, it is currently very difficult for policymakers to identify fast-grower firms before they become successful.
As demonstrated by the policy evaluations reviewed in Part II, many SME and entrepreneurship policies are selective, focusing upon sub-groups of the population. The first group of selective policies are those where eligibility is determined by identifiable characteristics of the enterprise, such as its sector, age, location, etc. This has the advantage that allocating policy support based on such criteria is a relatively simple administrative task.
A second group are those discussed above – enterprises thought likely to exhibit exceptional growth in the future. Identifying and then allocating funding to enterprises on the grounds that they will survive and expand considerably in the future is currently the most problematic, but potentially the most rewarding area for policymakers. It is problematic because there is a real risk that public funding of such enterprises may have little or no impact if the “wrong” firms are selected. However, it may be expected that, as more reliable evaluations are more frequently conducted, this knowledge will lead to better selections.
A third, and very important form of selection, is based on the personal characteristics of either the owner(s) or potential owner(s). These include policies to assist owners who are young, women, or from an ethnic minority, or individuals who are unemployed for example. Assistance to this group is intended to support “inclusive entrepreneurship” (OECD, 2013[7]) (OECD/European Commission, 2021[8]). Our review of 50 evaluations in Part II contained eight evaluations of inclusive entrepreneurship programmes, demonstrating that this policy area is amenable to reliable evaluations.
Recommendation: Evaluations should provide the evidence for making decisions on the scale and nature of selective support.
Furthermore, the presence or absence of a very small number of outstanding fast-grower firms among participants in a programme can have a strong influence on whether or not the programme is evaluated as having a positive impact, especially if it is a small-scale programme. However, this presence or absence can be affected by chance factors, given the infrequent probability of growers in the start-up population. Hence, the same result might not be achieved if the programme were repeated – as so much depends in a small-scale programme on the performance of the few. Not having a fast grower could make a programme appear as unsuccessful, whereas the inclusion of even a single fast-growing firm could make the programme appear successful.
It appears that none of the 50 policy evaluations reviewed in Part II took explicit account of the role of supported business that grew exceptionally fast in the overall impact result.
Recommendation: Evaluations should identify exceptional performers and the role such firms have in reaching a judgment on the overall effectiveness of a programme.
Governments seeking to raise the quantity and quality of small business and entrepreneurship activity in their country have, open to them, a range of policy options. Thus far we have reviewed “Micro”-based policies directed towards specific groups of individuals or enterprises that, because of market failures or social inequality, are thought to benefit from public support – either Hard, Soft or Both. The basic rationale for such support, as documented in Part I, is that this leads to increased economic activity or amongst the recipients in the form of sales and employment which, in turn, stimulates economic activity amongst others, so providing social benefits.
A second approach is the “Macro” approach. This seeks to raise the quantity and quality of entrepreneurship and SME activity in a country by focussing upon the economic, cultural and social environment – often referred to as the institutional framework – in which enterprises start and operate (Aidis, Estrin and Mickiewicz, 2012[9]). This approach sees the national institutional environment as a powerful influence on the individual’s decision to start a business and on the ability of the business to survive and grow. For example, the OECD/Eurostat Entrepreneurial Indicators Programme identifies the following key determinants of entrepreneurship in a country – the regulatory framework; research and development (R&D) and technology conditions; entrepreneurial capabilities; entrepreneurial culture; access to finance; and market conditions.1 Crucially, unlike Micro policies, it does not focus upon tightly-defined groups of enterprises or individuals, but rather on the business environment for all SMEs and entrepreneurs. The inference is that it is the nature of institutions which, positively or negatively, influence both the scale and nature of entrepreneurship (Autio and Rannikko, 2016[10]) which, in turn, influences economic welfare (Thornton, Ribeiro-Soriano and Urbano, 2011[11])38.
(Djankov et al., 2002[12]) were the first to identify and quantify the importance of one of the key institutional factors in the area of regulations – the costs and time taken to start a new (formal) business. They showed these were generally considerably higher in low-, than in high-, income countries. Since then, governments have sought to make it easier to “do business” by, for example, reducing the number of days required to register a business and lowering the costs of registration (i.e. a greater “ease” of doing business). Other institutional improvements have been introduced to assist SMEs and entrepreneurship by increasing the speed and efficiency of enforcement of legal and financial contracts. Improvements in governance are also expected to provide confidence to a potential new business owner.
Theorists, however, have argued that the scale and nature of entrepreneurship is influenced not only by formal, but also by informal institutions. By this they see belief systems, social norms and culture as powerful, albeit long-run, influences upon the willingness to start and operate an enterprise. A case has even been made that these exert a stronger influence than that of formal institutions (Thornton, Ribeiro-Soriano and Urbano, 2011[11]).
This institutional or Macro perspective offers potentially important, but not easy to accommodate, lessons for policymakers. This perspective – overlapping with the entrepreneurial ecosystem perspective – points to numerous factors influencing the scale and nature of entrepreneurship and SME development. Some are open to change in the short run, such as the cost and time taken to start a business. In contrast others – such as raising skills – can only be changed in the medium term. Finally, there are a group of potentially powerful influences that are extremely difficult to change, even in the longer run. There is currently no clear “route map” available to policymakers on how to change entrepreneurial culture as captured within the term “informal institutions”.
The core challenge for most policymakers is to bring about an identifiable impact during the electoral cycle. Inevitably this sees a concentration on the short- rather than medium- or long-run. Unfortunately, the evidence is that entrepreneurship rates change only slowly, except in the face of shocks such as macro-economic downturns (OECD, 2023, forthcoming[13]). Finally, many of the “easy wins”, such as regulatory improvements making it easier to start a business, have already been taken in recent years.
A further concern is that the science underpinning policy in this area remains under debate. For example, although there is a correlation between entrepreneurial culture and entrepreneurship activity it is less clear which causes which. For that reason, it is difficult to prescribe how policy makers should proceed when seeking to bring about a change in culture that, in turn, will raise entrepreneurship.
Because of the potential importance of all these issues, combined with the weak evidence base, there is a strong case for collaboration across governments. This might involve governments pooling their data and their knowledge on the influence of Macro issues and policies on entrepreneurship outcomes.
An example would be to review the full range of public services and examine their impact – both positive and negative – upon SMEs and entrepreneurship. It is important to include the full range of public services since some are not normally seen as SME relevant, yet can have a major impact on SMEs. A clear example is petty crime law-enforcement which has a considerable impact on SMEs (Drinkwater, Lashley and Robinson, 2018[14])]. High local crime rates, for example, can have a major negative influence on the ability of SMEs to trade productively.2
Recommendation: Governments should review the role played by “Macro” policies.
Indeed a case can be made that virtually all publicly-provided public services influence the scale and nature of entrepreneurship in a country or region. The education system in a country, for example, is frequently argued to influence subsequent employment choices of students – perhaps favouring low-risk employment over high-risk business ownership (Drinkwater, Lashley and Robinson, 2018[14]). For this reason many countries have implemented programmes to raise awareness of entrepreneurship amongst those yet to enter the labour force (Fayolle, 2013[15]).
References
[4] Acs, Z. et al. (2016), “Public policy to promote entrepreneurship: a call to arms”, Small Business Economics, Vol. 47/1, https://doi.org/10.1007/s11187-016-9712-2.
[9] Aidis, R., S. Estrin and T. Mickiewicz (2012), “Size matters: Entrepreneurial entry and government”, Small Business Economics, Vol. 39/1, https://doi.org/10.1007/s11187-010-9299-y.
[10] Autio, E. and H. Rannikko (2016), “Retaining winners: Can policy boost high-growth entrepreneurship?”, Research Policy, Vol. 45/1, https://doi.org/10.1016/j.respol.2015.06.002.
[6] Daunfeldt, S. and D. Halvarsson (2015), “Are high-growth firms one-hit wonders? Evidence from Sweden”, Small Business Economics, Vol. 44/2, https://doi.org/10.1007/s11187-014-9599-8.
[12] Djankov, S. et al. (2002), “The regulation of entry”, Quarterly Journal of Economics, Vol. 117/1, https://doi.org/10.1162/003355302753399436.
[14] Drinkwater, S., J. Lashley and C. Robinson (2018), “Barriers to enterprise development in the Caribbean”, Entrepreneurship and Regional Development, Vol. 30/9-10, https://doi.org/10.1080/08985626.2018.1515821.
[15] Fayolle, A. (2013), “Personal views on the future of entrepreneurship education”, Entrepreneurship and Regional Development, Vol. 25/7-8, https://doi.org/10.1080/08985626.2013.821318.
[16] Ganau, R. and A. Rodríguez-Pose (2018), “Industrial clusters, organized crime, and productivity growth in Italian SMEs”, Journal of Regional Science, Vol. 58/2, https://doi.org/10.1111/jors.12354.
[17] OECD (2019), OECD SME and Entrepreneurship Outlook 2019, OECD Publishing, Paris, https://doi.org/10.1787/34907e9c-en.
[7] OECD (2013), “Entrepreneurial Activities in Europe: Evaluation of Inclusive Entrepreneurship Programmes”, OECD Employment Policy Papers 4, https://doi.org/10.1787/5jxrcmkm81th-en.
[3] OECD (2007), OECD Framework for the Evaluation of SME and Entrepreneurship Policies and Programmes, OECD Publishing, Paris, https://doi.org/10.1787/9789264040090-en.
[13] OECD (2023, forthcoming), “Leapfrogging and Plunging in Regional Entrepreneurship Performance: US and European Comparison”, OECD SME and Entrepreneurship Papers.
[8] OECD/European Commission (2021), The Missing Entrepreneurs 2021: Policies for Inclusive Entrepreneurship and Self-Employment, OECD Publishing, Paris, https://doi.org/10.1787/71b7a9bb-en.
[5] Parker, S., D. Storey and A. van Witteloostuijn (2010), “What happens to gazelles? The importance of dynamic management strategy”, Small Business Economics, Vol. 35/2, https://doi.org/10.1007/s11187-009-9250-2.
[2] Rotger, G., M. Gørtz and D. Storey (2012), “Assessing the effectiveness of guided preparation for new venture creation and performance: Theory and practice”, Journal of Business Venturing, Vol. 27/4, https://doi.org/10.1016/j.jbusvent.2012.01.003.
[1] Sara, R. (2016), Start-Up Support for Young People in the EU: From Implementation to Evaluation, Eurofound.
[11] Thornton, P., D. Ribeiro-Soriano and D. Urbano (2011), “Socio-cultural factors and entrepreneurial activity: An overview”, International Small Business Journal, Vol. 29/2, https://doi.org/10.1177/0266242610391930.
Notes
← 1. The OECD SME and Entrepreneurship Outlook (OECD, 2019[17]) similarly focuses on a set of national institutional conditions affecting SME performance – namely institutional and regulatory framework; market conditions; infrastructure; access to innovation assets; access to skills; and access to finance.
← 2. This has been most extensively documented in Italy [ (Ganau and Rodríguez-Pose, 2018[16])]