Foreword
The purpose of this book is to support and encourage policy makers to develop reliable evaluations of the impact of SME and entrepreneurship policies and to make use of impact evaluation results in policy design and implementation. The book argues that systematic and reliable evaluation is vital for justifying the use of public resources for SME and entrepreneurship support and for steering those resources to the policy measures that deliver the greatest benefits against government objectives. It offers government policy makers guidance in making the case for evaluation, commissioning evaluations that will be reliable and learning from existing and new evaluations.
The starting position of the book is that there remains a dearth of reliable impact evaluation evidence in the domain of SME and entrepreneurship policies, despite great improvements in data availability and evaluation methodologies since the previous version of this Framework appeared in 2007. This gap needs to be addressed through more and better evaluation. The book provides recommendations and guidance on how to build the required evidence base. At the same time, however, there is a small core of high-quality SME and entrepreneurship policy impact evaluations internationally, ranging across different OECD countries and different policy intervention areas. This book profiles the methodologies and findings of a selection of these high-quality evaluations. The findings are mixed in terms of the extent to which the evaluations report positive policy impacts on key policy objectives. The book considers the implications of this mixed evidence for future policy development.
The book is part of the programme of work of the OECD Committee on SMEs and Entrepreneurship (CSMEE), which continues the work of its predecessor, the OECD Working Party on SMEs and Entrepreneurship (WPSMEE) in this area. Promoting SME and entrepreneurship policy evaluation has long been a priority for these bodies, as evidenced by the calls for better evaluation in the declarations of each of the OECD SME and entrepreneurship ministerial meetings – Bologna (2000), Istanbul (2004), Mexico City (2018) – and in the Recommendation of the OECD Council on SME and Entrepreneurship Policy, issued in 2022.
The book highlights several issues that need to be addressed by the SME and entrepreneurship policy community. Its overall concern is that the evaluation evidence on the impact of SME and entrepreneurship policy remains weak, and it considers possible reasons why policy makers may not be pursuing reliable evaluation and how to respond to this. It also signals a number of obstacles to reliable evaluation that policy makers need to address, such as insufficient clarity on policy objectives or lack of control group data to establish a counterfactual. It offers guidance on how policy makers can address such issues.
The book also includes profiles of 50 high-quality impact evaluations of SME and entrepreneurship policy in OECD countries. This does not by any means include all such evaluations from recent years, but it does provide a range of important examples from a variety of OECD countries and across different policy areas. The book profiles the methodology used for each evaluation, which can serve as inspiration for other evaluators and the policy makers commissioning evaluations. It also profiles the key findings of each evaluation. By gathering together only reliable findings, we can start to consider what works and does not work in SME and entrepreneurship policy. While this book can only touch the surface of this ongoing endeavour, it does provide some important hypotheses for exploration with further evaluation work.
As well as offering a self-contained set of information and guidance, this Framework serves as a foundation of OECD tailored support to countries for strengthening their SME and entrepreneurship policy monitoring and evaluation arrangements. Thus, the key considerations for successful evaluation set out in this volume are taken up in one-to-one OECD advice, where requested by governments, on creating effective policy monitoring and reporting systems and in deciding on potential methodologies for evaluating their major programmes.
The volume updates the previous OECD Framework for the Evaluation of SME and Entrepreneurship Policies and Programmes (OECD, 2007[1]) in order to take account of recent developments in both the techniques and data sources available for SME and entrepreneurship policy evaluation and in the scope and practice of SME and entrepreneurship policy.
It forms part of broader OECD efforts to strengthen policy evaluation across government as a whole. In particular, it will support the implementation of the OECD Recommendation on Public Policy Evaluation [OECD/LEGAL/0478] and the OECD Recommendation on SME and Entrepreneurship Policy [OECD/LEGAL/0473], both adopted by the OECD Council in June 2022.
This document was approved by the OECD Committee on SMEs and Entrepreneurship (CSMEE) through written procedure on 09 December 2022 (CFE/SME(2022)23) and prepared for publication by the OECD Secretariat.