Policy framework and assessment process
The SME Policy Index is a benchmarking tool designed to assess policies that support small and medium-sized enterprises (SMEs) in emerging economies and monitor progress in policy implementation over time. The index was developed in 2006 by the OECD in partnership with the European Commission, the European Bank for Reconstruction and Development (EBRD) and the European Training Foundation (ETF). Since then, it has been applied 10 times, covering 33 economies in 4 regions: the Western Balkans and Turkey (WBT), Eastern partnership countries, North African and Middle East regions, and the Association of Southeast Asian Nations countries. For the WBT region, it is structured around the ten principles of the Small Business Act for Europe (SBA), providing a wide range of pro-enterprise measures to guide the design and implementation of SME policies in the European Union.
While a number of other indices and benchmarking reports have been used to assess the business environment in the WBT region, the SME Policy Index follows a holistic approach that provides policy makers with an analysis of their strengths and weaknesses in SME-related policy settings, allowing comparisons to be made across economies and measuring convergence toward OECD and EU good practice1. The report also monitors alignment in enterprise policy with the EU acquis, especially with respect to Chapter 20, and provides inputs into the Economic Reform Programmes (ERPs), most notably under the business environment structural dimension.
The SME Policy Index is divided into 12 regional policy chapters and 7 economy-specific profiles, which contain individualised structural reform recommendations tailored to the specific challenges of each WBT economy.
The overall assessment approach
The SME Policy Index: Western Balkans and Turkey 2022 records each economy’s progress across a framework of 12 policy dimensions. Each policy dimension assesses SME and entrepreneurship-enhancing policies through up to five sub-dimensions, further broken down into thematic blocs, employing qualitative and quantitative indicators (Figure 2). The set of indicators used for each of the 12 policy dimensions can be found in the “assessment framework” included at the beginning of each regional chapter. The 2022 report includes 34 sub-dimensions, 88 thematic blocks and more than 700 indicators.
The SME Policy Index assessment framework was developed to respond to the priorities identified in the ten principles of the SBA. Table 4 shows how the policy dimensions and sub-dimensions relate to each SBA principle.
Refinements to the 2022 assessment framework
The SME Policy Index 2022 has been revised since the 2019 assessment to provide more in-depth analysis and better respond to the needs of the economies being assessed. Refinements to the present edition have the following main objectives:
assess the impact of the COVID-19 pandemic on different policy areas and better inform policy making during and in the aftermath of the pandemic
provide inputs for national SME development strategy and other national development strategies (e.g. innovation, education, environmental protection) as well as guide the ERPs by shedding light on the most pressing policy challenges hampering the business environment
increase the assessment’s focus on analysing the implementation of the policy recommendations provided in the previous SME Policy Index edition
shed light on what the WBT governments should do to further their development agendas both in the post-COVID (“build back better”) and EU accession contexts (Berlin process, Economic and Investment Plan for the Western Balkans, Green Deal and Green Agenda for the Western Balkans, Digital agenda, etc.), by incorporating the themes that are becoming of increasing importance for the region (e.g. environment, digitalisation)
review and embed the latest OECD instruments, tools and good practices as well as the dominating policy trends in OECD and EU economies as per their relevance
introduce, when relevant, new indicators based on these trends while ensuring that comparable data are available across economies
take into account limitations related to the availability of data and other available data sources, i.e. only indicators that allow for benchmarking are included.
Qualitative indicators
Each qualitative indicator is assigned a numerical score that reflects the level of policy development, implementation and monitoring to facilitate the comparison of performance among the WBT economies. Qualitative indicators are given a score from 1 to 5, with Level 1 being the weakest and Level 5 the strongest (Figure 3). The results are then aggregated with weightings applied at the sub-dimensional and thematic block levels, taking into consideration the importance of each indicator in policy formulation or implementation. A detailed description of the allocation of scores is presented in Annex A.
Quantitative indicators
In order to make it easier to compare performance across economies and over time, quantitative indicators are used to provide additional evidence on the performance of policy settings, processes and programmes. These include horizontal statistical data to better understand the nature of the SME sector; and data specific to individual dimensions to assess policy outputs and outcomes relevant for implementation. Although the quantitative data are not included in the scoring, they have been taken into account in the narrative text, including the analysis of the overall SME policy setting and formulation of policy recommendations. The OECD primarily collected the quantitative data from national statistical offices.
The SME Policy Index 2022 is based on the results of two parallel assessments: the economies’ governments’ self-assessment, which involved completing a questionnaire and assigning a score for each policy indicator using the assessment grid; and the OECD and its partner organisations’ independent assessment, with inputs from a team of local experts.
The final scores are derived via a highly collaborative and consultative consolidation of these two assessments and enriched by consultations with key stakeholders (e.g. private sector, business associations, academia and civil society) in the participating economies.
Assessment phases and timing of the 2022 assessment
The assessment was carried out between March 2021 and July 2022 in four main phases: 1) design; 2) data collection and evaluation; 3) consolidation; and 4) review and publication (Figure 4).
Specifically speaking, the phases can be described as follows:
Design phase (March 2021-June 2021). The OECD and the partner institutions revised the assessment framework in consultation with the SBA Co-ordinators, partner organisations and the European Commission, to ensure that the latest international and OECD good practice had been incorporated into the assessment frameworks for each policy dimension.
Data collection and evaluation phase (July 2021-October 2021). In light of the COVID-19 pandemic (Box 3), a regional launch meeting was held virtually to present the new assessment framework to the seven economies’ line ministries and public institutions that were expected to contribute to the information collection process. Explanatory meetings were also organised with SBA Co-ordinators and statistical offices, in which the two documents making up the assessment framework – the questionnaire and statistical data sheet – were explained in depth to the participants, directing particular attention to the newly added questions.
Following the launch events, the SBA Co-ordinators distributed the questionnaires to the appropriate counterparts in the ministries and government agencies, as well as the statistical sheet to the national statistical offices. The SBA Co-ordinators then compiled the data and documentation and completed the questionnaire. Scores were assigned for each policy dimension, with an accompanying justification. The completed questionnaires and statistical data sheets were sent to the OECD team by October 2021. From mid-September, the OECD and its partner organisations conducted an independent assessment, supported by a team of local experts who collected additional data and information, as necessary.
Consolidation phase (November 2021-March 2022). Reviews of the inputs by the OECD and the partner institutions revealed the need for additional information on certain elements. Some 30 virtual consultation meetings with over 100 stakeholders were held to fill in the remaining information gaps in November 2021.
Once the data collection was complete, the OECD organised virtual economy-specific meetings. These meetings aimed to present the preliminary findings and recommendations and close any remaining information gaps in the questionnaire, discussing discrepancies between the two parallel assessments and filling in missing information.
While these meetings were held in the six Western Balkan and Turkey capitals in previous assessment cycles, COVID-19 restrictions meant that the roundtables were held virtually.
After the economy-specific meetings, the assessment findings were consolidated. The OECD and partner organisations decided on the final scores under each policy dimension presented in this report. The regional preliminary findings and scores were subsequently presented to the seven economies in a virtual regional meeting on 3 March 2022.
Review and publication phase (March 2022-July 2022). The draft SME Policy Index 2022 chapters and the related SBA economy profiles were made available to the WBT governments, the European Commission and the EU delegations for review during February-April 2022. Meanwhile, the draft report was also peer-reviewed internally by the OECD, the EBRD and the ETF. Following the review, final comments were integrated into the report to be launched at the Dubrovnik Forum on 8 July 2022.
The COVID-19 pandemic triggered travel restrictions worldwide, requiring a novel approach to this SME Policy Index assessment cycle. In light of these extraordinary developments, the OECD team adopted the following measures:
Virtual launch of the assessment: All assessment materials were transformed into a digital format, making them accessible and easy to understand and use for all those involved in the assessment. This made it possible for multiple persons to simultaneously work inside the assessment materials, enabling co-ordinators to fill them out and review inputs simultaneously.
Virtual meetings: In lieu of physical meetings at which the assessment would be explained in depth, and questions answered once and for all, the OECD team instead held individual virtual meetings, as well as follow-up calls with the relevant stakeholders in the WBT economies. The presentations of preliminary key findings on regional and economy-specific levels were also conducted using virtual meeting platforms, which allowed for a higher level of participation.
Despite these measures, the challenges created by the COVID-19 pandemic caused some delays in the input collection process. A number of assessment contributors contracted COVID-19 and needed to be temporarily replaced, which negatively affected the co-ordination across various administrative units and slowed the government self-assessment data-collection process.
The SME Policy Index: Western Balkans and Turkey 2022 possesses several strengths that make it a uniquely valuable report for policy makers, citizens, researchers and international development co-operation partners. Nevertheless, it also has some limitations that need to be considered (Table 5).
Note
← 1. The newly-adopted Recommendation of the Council on SME and Entrepreneurship Policy aims to provide an evidence-based and holistic framework to support OECD member states in developing coherent, effective and efficient SME and entrepreneurship policies can also be used as a reference in SME policy-making in Western Balkans and Turkey. For more information, see https://www.oecd.org/cfe/smes/oecdrecommendationonsmeandentrepreneurshippolicy/