3. Evidence-informed Decision-making in the Czech Administration
Taking decisions informed by evidence can help governments to achieve higher living standards for its citizens. The chapter critically examines the current conditions of evidence-informed decision-making in the Czech Republic. It highlights that the demand for evidence-informed decision-making ultimately originates at the political level. Stakeholders play a crucial role in both providing evidence to improve decision-making, as well as scrutinising evidence used as a means to improve government accountability. The ability to gather evidence is determined by data availability and accessibility. Using evidence to inform decision-making depends on the analytical capacities within the civil service, as well as appropriate institutional arrangements. The chapter provides specific recommendations based on international experiences to lead to an improved decision-making environment in the Czech Republic.
Evidence-informed decision-making (EIDM) is a process whereby multiple sources of information, including statistics, data and the best available research, evidence and evaluations, are consulted before taking a decision to plan, implement, and (where relevant) alter public policies and programmes (OECD, 2020[1]). The goal of EIDM is to enable better public policy decisions, which, among other things, reduces the risk of regulatory failures and promotes dialogue and contestability with regulated entities. Using EIDM can help achieve broad societal goals, such as increasing trust in government and decision-making, promoting sustainable development, and improving well-being. It also strengthens and improves public finances, as EIDM helps to allocate scarce public resources to value-for-money activities, and can assist in prioritising public expenditures and worthwhile reforms.
EIDM is particularly useful to inform the policymaking process, for instance, to enable an assessment of the likely impacts of regulatory proposals. Its usefulness stems from its protean nature: “When my information changes, I alter my conclusions. What do you do, sir?” (Crowley, 1986[2]). An alternative way of considering Keynes’ retort is that an absence of information results in the maintenance of the status quo. Moreover, the situation remains unchanged until new information is forthcoming. As such, information ought to be the driving force for improved decision-making throughout public administrations.
The “market” for evidence is characterised by both demand and supply factors. Political leadership is essential to ensuring policymakers understand how evidence will be used in shaping decisions. Not only do decision-makers have a greater appreciation for the policy choices put forward, but EIDM can help improve the communication on and implementation of policies. Political leadership is also intrinsically linked with the supply of evidence. For instance, obtaining information and evidence is not a costless exercise, and scarce public resources should be allocated to their highest value use. If the demand for evidence is sufficient, then civil servants need appropriate skills, training and resources to carry out EIDM.
Overcoming information and data availability issues is important to attain EIDM. Yet, we live in the information age with more data than ever before. The advent of new technologies has meant that some actions can now be monitored for the first time, and others can now be either better monitored or be done so at lower costs. What matters, though, is the quality of the data. Additional supply elements are the extent to which data are provided openly to the public and the extent to which data systems are designed with interoperability in mind.
The information generated is dependent on the civil service's organisational capacities. At a corporate level, this means having the necessary architecture to support civil servants to use evidence for policymaking. The extent to which EIDM becomes a reality is, in part, determined by the skills of the civil servants themselves. Their skills include those they have when joining the civil service as well as those that they accrue through training and development programmes.
EIDM is unlikely to be held in high regard by senior civil servants tasked with carrying out decision-makers’ wishes if political demand is lacking. Additionally, civil servants may be reluctant to provide information to decision-makers if they think it will merely justify a decision already taken rather than helping to form the basis for decision-making. Overcoming inertia is crucial for EIDM to take hold in the civil service.
Additional demand for EIDM comes from stakeholders, including businesses, citizens, academia and non-governmental organisations (NGOs). These entities are, of course, an important source of evidence in their own right. Additionally, they can provide an important quality assurance role of the evidence supplied to decision-makers based on their own learnt experiences.
Next comes the demonstrative actions of political will in the form of institutions. These actors play a critical role in both setting and enforcing standards for policymakers to undertake EIDM. Then there are players outside of the government sphere. Not only do stakeholders have a wealth of information, but they can also hold governments to account through participatory democracy. It is through this that they can perpetuate continued demand for high and improved evidence-informed decision-making standards.
Government activity partially underpins the demand for EIDM. For example, guidance materials, methodological documents and so on provide the foundations for civil servants to understand why EIDM is important, what it is and how it can be incorporated into their daily work. Having a framework for EIDM is indeed necessary, but is not sufficient. For EIDM to take hold, it needs to be implemented.
Regulatory management tools, including RIA and ex post assessment of regulations, play a strong complementary role to EIDM. RIA is a process of policymaking whereby the likely impacts and consequences of various regulatory and non-regulatory options are considered and evaluated to inform decision-making (OECD, 2018[3]). RIAs help to substantiate decisions, not to intervene in markets where the costs are too high relative to the benefits, or to communicate the arguments when such intervention is found necessary. They ensure that the impact on a particular societal or interest group that might be marginalised or absent from the mainstream public debate is assessed (OECD, 2020[4]).
Ex post assessments are systematic reviews of the stock of regulation against clearly defined policy goals, including consideration of the costs and benefits, to ensure that regulations remain up to date, cost-justified, effective and consistent, and deliver the intended policy objectives. Ex post assessments complete the “regulatory cycle” that begins with an ex ante assessment of proposals and proceeds to implementation and administration (OECD, 2018[3]). Well-established practices and processes of ex post assessments can ensure the effectiveness, value-for-money, accountability and transparency of policymaking. Such reviews allow the government to identify the effects of existing legislation and can help to improve the design and administration of new regulations (OECD, 2020[1]).
Demand for EIDM is ultimately created by politicians. If there is a clear and demonstrated appetite for decisions to be informed by evidence, then resourcing (including training) and senior civil servant culture should follow. Of course, various stakeholders – businesses, citizens, academia, NGOs, etc. – also play a crucial role in demanding EIDM from a political accountability point of view. It should be noted that stakeholders are also a source of evidence based on their own learnt experiences and can help policymakers in their search for evidence, as well as in improving its quality. Institutions play a vital role in signalling that there is both political and senior official appetite to engage in EIDM. The existence of guidance material and manuals all help to establish the importance of EIDM and how it can be transmitted to civil servants on the ground.
Political support
Political support is crucial for EIDM to succeed. Without it, there will be little top-down signalling to civil servants, businesses and the broader public that EIDM is important and that all parties should work together to gather better data, engage with stakeholders and test ideas to take more informed decisions.
Governments decide either explicitly or implicitly on the resource allocation and priorities of the public administration through, for example, the budgetary process, ministerial statements, etc. The government sends an important signal to senior civil servants about where scarce public resources should be allocated. Governments, as the executive, are also responsible for the vast majority of laws that pass through national parliaments. However, it should be noted that the Czech Republic has recently been an exception in this regard (OECD, 2021[5]). In instances where legislative agendas are heavy, and there are expectations that governments will act quickly, there can often be insufficient time for EIDM. An absence of EIDM increases the risk of regulatory failure, poorly designed and/or implemented laws, and in the end, can do more harm than good (OECD, 2018[3]). In part, it is why the OECD has advocated for legislative plans to be drawn up, but this does not imply that EIDM is inherently inflexible when genuine unforeseeable emergencies call for swift government action. Recent country examples highlighted a range of operational flexibilities in rule making during the height of the COVID-19 pandemic. Another commonality was that these rules tended to be subject to future repeal (i.e. sunset clauses) or had an in-built review requirement because of the general lack of EIDM (OECD, 2021[5]).
In its Policy Statement of the Government, the current Czech government promised to “carefully consider every new regulation based on an analysis of the expected impacts” (Government of the Czech Republic, 2022[6]). It also mentions that “for every new law, government decree or bylaw its functioning in practice will be evaluated at the latest 5 years after entering into force” (Government of the Czech Republic, 2022[6]).
Since the creation of the current coalition government in 2021, political responsibility for regulatory reform in the Czech Republic resides with the minister for legislation and the chair of the Government Legislative Council. The functions performed have, however, remained unchanged and include:
monitoring and reporting on the co-ordination of regulatory reform activities across portfolios
reporting on the performance of the regulatory management system
identifying opportunities for improvement to regulatory policy settings and regulatory management practices (OECD, 2021[7]).
The Czech Republic currently has a number of high-level documents with EIDM aspects (see below). All of them have been endorsed by the government, which suggests that there is some level of political support for EIDM. At the same time, however, it is recognised that political commitment to EIDM remains fragile. The Client-oriented Public Administration 2030 sets that, in 2030, authorities should “generate innovative solutions and make responsible decisions based on data evidence.” While this is a positive development, it is clear that fundamental risks threaten its attainment, and indeed the Ministry of Interior has identified many. Moreover, the potential risks turned out to be real and valid barriers. Chief among them are political risk, government negotiations with the parliament and civil service inertia (Box 3.1).
The Ministry of Interior identified 36 project risks to fulfilling the objective of the Client-oriented Public Administration 2030 (“the Concept”), classified as either very significant, moderate or low. The very significant risks identified are, in descending order of importance:
failure to enforce the necessary legislative changes when agreement cannot be found in the Chamber of Deputies and the Senate
insufficient financial assurance of the implementation of the Concept and its action plans
the emergence of low-quality outputs that are not applicable (in part or full) or have no added value
the public administration’s resistance to change and the reluctance of government stakeholders to accept the proposed changes.
The risk register further highlights a series of moderate evidence-informed decision-making risks, including:
the government’s non-compliance with the schedule of activities and measures in the action plan
insufficient or insufficiently high-quality personnel to ensure the fulfilment of the objectives of the Concept and its action plans
complex analysis processing where suitable methodologies may be difficult to identify, data may not be available and international comparisons may be fraught
non-acceptance of outputs by interested parties, including within the department
low-quality or insufficient management of the implementation of the Concept
inappropriate procedure for achieving goals (e.g. important facts were omitted during the preparation, activities were chosen to fulfil the goals and measures that cannot contribute to the fulfilment of the goal or can only partially contribute, an appropriate methodology was not chosen for implementing activities, etc.).
Source: Ministry of the Interior of the Czech Republic (2019[8]).
This chapter highlights several areas where political commitment could be strengthened to enhance EIDM in the Czech Republic. First, demand for EIDM by decision-makers and senior civil servants could be improved. EIDM is a tool to help explain why government intervention is necessary and demonstrate that such intervention is in the public’s interest. EIDM can therefore help inform stakeholders and the broader public about proposed government action. It can also help to choose the most effective and efficient solutions for implementing the government’s strategic goals. However, it is still generally the case that, when used, EIDM in the Czech Republic is largely to justify a decision that has already been taken rather than to help inform the decision-making process. Decision-makers’ expectations that the civil service will deliver evidence are of critical importance to ensure that EIDM takes hold. Second, as a result, more investment in civil service capacities is needed to make EIDM a reality in the Czech Republic. It entails taking political decisions about where to allocate scarce public resources. However, in the absence of EIDM, decision-makers’ expectations will largely go unmet, given the current level of investment. Third, decision-makers can demonstrate demand for EIDM by ensuring policy coherence and complete implementation and review. Additional improvements can be made to the policymaking processes. In particular, impact assessment, consulting with stakeholders, the composition and functions of the regulatory oversight body (which is responsible for checking the quality of prospective laws), and instituting a compulsory process of ex post regulatory reviews. Improvements in these areas would demonstrate strengthened political commitment towards EIDM.
Stakeholder engagement
Stakeholders play an essential role in demanding EIDM. Broadly, there are connections with notions of political accountability and the extent to which elected officials are held accountable by state or civil society organisations, including the media, for the quality of their decision-making (OECD, 2020[1]). The extent to which there is a culture of inquiry and how this is developed through institutions such as higher education also determines the extent to which evidence is seen to be an important input to the policymaking process (Newman, Fisher and Shaxson, 2012[9]).
Citizens can offer valuable inputs on the feasibility and practical implications of regulations (see Chapter 1). Meaningful stakeholder engagement can lead to higher compliance with regulations, in particular when stakeholders feel that their views have been taken into consideration (OECD, n.d.[10]). From a regulatory policy perspective, this entails offering the public sufficient opportunity to help shape, challenge and reform the regulations they encounter in their daily lives (OECD, 2021[5]).
It should be noted that stakeholders have an influence on the supply of evidence. First, societal attitudes towards policymaking, and what and who should contribute to it, can impact the use of evidence in policymaking (Newman, Fisher and Shaxson, 2012[9]). Second, stakeholders are a rich source of information, both in terms of data and through their learnt experiences dealing with previous and/or current regulations. As they have the potential to affect both the demand and the supply of evidence, stakeholders can play a highly influential role in determining the extent of EIDM – assuming that policymakers allow such engagement to take place.
Generally, consultations should be available to all citizens (OECD, 2017[11]). Beyond that, specifically determining who to consult effectively means deciding who should be excluded from the consultation process. There may be justified reasons for limiting consultations due to factors such as confidentiality, the subject nature of the proposal (e.g. if it is highly technical or if expertise lies only in limited areas) and for genuine matters of expediency (although this should not be used as a default excuse to avoid consulting) (OECD, 2021[5]).
Core aspects of stakeholder engagement are either currently missing or underutilised in the Czech Republic. At a fundamental level, no general requirement exists that consultations must be open to the general public, contrary to more than half of OECD countries that have systematic requirements in place (Figure 3.1) (OECD, 2021[7]).
Notes: * In the majority of OECD countries, most primary laws are initiated by the executive, except for in Colombia, Costa Rica, the Czech Republic, Korea, Mexico and Portugal, where a higher share of primary laws are initiated by the legislature. Due to a change in the political system during the survey period affecting the processes for developing laws, composite indicators for the Republic of Türkiye are not available for stakeholder engagement in developing regulations and regulatory impact assessment for primary laws. Data for 2014 are based on the 34 countries that were OECD countries in 2014 and the European Union. Data for 2017 and 2021 include Colombia, Costa Rica, Latvia and Lithuania. As advocated in the 2012 Recommendation of the Council on Regulatory Policy and Governance, the more regulatory practices a country has implemented, the higher its iREG score. The indicator only covers practices in the executive. This figure, therefore, excludes the United States, where all primary laws are initiated by Congress.
Source: OECD (2021[5]).
Without some minimal degree of standardisation in stakeholder engagement, ministries have evolved divergent practices. For instance, some ministries systematically inform stakeholders in advance of forthcoming consultations while others do not. This not only can lead to divergent practices across ministries, but also to divergences across units within ministries. Ultimately, policymakers therefore have significant discretion in whether, and to what extent, to seek input from stakeholders. While discretion allows for significant operational flexibility – recognising, for instance, that not all proposals necessarily need broad public consultation, that it will not be appropriate in some instances (e.g. matters relating to national security or other highly sensitive proposals) and that consultation fatigue among stakeholders does exist, so a degree of tailoring is appropriate – there does not currently exist any commensurate level of oversight in the decision taken.
One co-ordination issue is that ongoing consultations are available only via a ministry’s web pages and are currently not centralised. A centralised portal aim of which would be to ensure public consultations does not exist for the moment. The eKlep system enables public comments, and various ministries’ comments on regulatory proposals can be viewed. However, its interface is not user-friendly and due to the low awareness of this platform among the general public, external stakeholders very rarely submit comments. The system could be relatively easily used as a basis for an interactive consultation portal, which is becoming more commonplace across the OECD (OECD, 2021[5]). Not only would this improve and better target engagement with stakeholders, it would also help to systematically ensure that all draft regulatory proposals were open to public consultation. In addition, it would improve the quality of engagement: stakeholders would be able to see and challenge the veracity of claims made by others. This process could help policymakers better estimate the likely impacts and thereby reduce the risks of regulatory failure.
It is important to allow for sufficient checks and balances within a consultation process. For instance, there is a risk that consulting the “usual suspects” leads to the “usual answers”. Policymakers can be assisted in identifying vested interests (and thereby reducing risks of regulatory capture) by consulting broadly, allowing other stakeholders to challenge positions put forth by the “usual suspects” (OECD, 2012[12]). For this, it is necessary for administrative authorities to carefully map which groups of stakeholders might be affected by the policy or regulation and proactively reach out to them to engage them in the consultations rather than waiting for stakeholders to volunteer. The DataKO database is a voluntary list of organisations that can be contacted in the case of ongoing consultations in a matter related to their work/area of expertise. It could be used to help ministries identify potentially impacted stakeholders (Government of the Czech Republic, 2016[13]). However, it should be noted that stakeholders registering on such platforms are more likely to be part of the “usual suspects”, so it is likely that complementary engagement will be required to ensure that all stakeholders have the opportunity to participate. The level of stakeholder engagement varies across ministries. For example, the OECD heard of strong engagement between the Ministry of the Environment and NGOs, but engagement was not so systematic with the business community.
Engagement refers to two-way communication between the government and stakeholders. One important aspect of engagement is the processes that governments have in place to illustrate how input received has helped to shape regulatory proposals or to note why particular suggestions were not taken on board. Guidance on conducting stakeholder engagement in the Czech Republic is available for civil servants.1 However, in contrast to more than 60% of OECD countries, it does not require policymakers to consider consultation comments when making draft regulations (OECD, 2021[7]). Such an approach can frustrate stakeholders, who are left in the dark about the extent to which their input has helped shape regulatory proposals. It reflects a wider absence of a consultation culture in the Czech administration, where there are essentially no consequences for failing to apply the policy development guidelines.
Deciding when to consult is a central facet of decision-making. There are generally four distinct stages of consultation: to inform the community in advance; at the early and late stages of policy development; and on the revision and modification of existing laws (OECD, 2012[12]). The 2030 Strategic Framework for the Czech Republic recognises this: deliberations can take place at all phases of the policymaking process (Government of the Czech Republic, 2017[14]). Establishing when to consult can be of critical importance to the design of the resultant policy: if consulted too early, stakeholders may not be able to help identify potential solutions; if consulted too late, they may feel that consultation is an obligatory step for policymakers to progress their policies to the decision-making stage (OECD, 2021[5]).
Like most OECD countries, stakeholders are not systematically informed in advance of forthcoming consultations in the Czech Republic (OECD, 2021[5]). Once consultations have commenced, stakeholders are rarely called upon to provide input. The early stages of policy development are a critical juncture to gather information from stakeholders about the potential magnitude of identified policy problems and elicit potential solutions (OECD, 2021[5]). It also presents an opportunity for ministries to learn more about potential areas or sectors that may be affected and that warrant in-depth impact analysis. Stakeholders are usually provided with either a general consultation document describing the problem and soliciting input on possible solutions (e.g. via a questionnaire) or some other sort of analytical document or study. It is interesting to note that despite the mandatory completion of the overview of impacts in the Czech RIA Guidelines (see below) (Government of the Czech Republic, 2016[13]), no overview is ever issued for consultation (OECD, 2021[7]). Stakeholders may be able to assist policymakers at this stage based on the overview to determine whether impacts are expected in specific areas that ought to then undergo more in-depth analysis, in conformity with the proportionality principle in the Czech Republic (Government of the Czech Republic, 2016[13]).
As in most OECD countries, consultations most frequently occur in the Czech Republic once a draft regulation exists (OECD, 2021[7]). However, consultations are most likely to occur with selected groups of stakeholders and do not necessarily invite comments (OECD, 2021[7]). This reflects the overall nature of stakeholder engagement in the Czech Republic. While consultation at a later stage of policy development may be more focused, it is important to make sure that alternative views are given an adequate opportunity to comment, especially in instances where they were not involved at a more nascent stage (OECD, 2021[5]). A broader range of consultation material is made available to stakeholders at a later stage of policy development and can include:
Even though RIAs may occasionally be consulted on, there is no formal requirement to do so in the Czech Republic (OECD, 2021[7]). This again highlights the level of discretion individual policymakers have within the current system. Consulting on the RIA allows for various assumptions to be queried, for the impact assessments to be verified or contested by affected parties, and for alternatives to be put forward (OECD, 2020[4]). In short, it improves the evidence base upon which more informed decisions can be taken.
Stakeholders should be involved in evaluating regulations (OECD, 2020[15]). They can assist policymakers in assessing the actual regulatory impacts “on the ground”. They have a potential wealth of information about the actual impacts and how these may differ across affected parties. The information gleaned can help improve the policy over time to ensure it remains fit for purpose. After all, decision-making is dynamic, not static, reflected in the fact that all laws are experiments and sometimes do not work out as originally envisaged. Stakeholders can also assist policymakers by highlighting areas of the law that are not working as originally intended and can suggest improvements. Stakeholder engagement during ex post assessments also helps to maintain awareness and understanding of the rationale for the regulation. However, the public is not informed in advance of forthcoming ex post assessments and is only invited to be involved in some reviews (OECD, 2021[7]).
One indicator under the measurement and evaluation of the Czech public administration (see below) relates to citizen satisfaction with the legal environment. It is important to seek the views of affected stakeholders about the legal environment, although a number of methodological issues arise when undertaking perception surveys, particularly those of citizens (OECD, 2012[16]). The annual report data are based on a sample survey conducted by the Public Opinion Research Centre, and the results are an average of those responses. It is not clear to what extent the surveyed citizens are asked questions about the legal environment or, for example, whether questions are asked on specific aspects (e.g. regulatory design, implementation, enforcement, etc.) and/or specific areas (e.g. environment, health, transport, etc.). Irrespective of the merits of the indicator used, no target is specified. All the annual report mentions is that an upwards trend is desired (Ministry of Interior of the Czech Republic, 2022[17]). Assuming that the survey is relatively statistically robust, it would seem more appropriate to have a quantitative target set (e.g. a representative majority of citizens are satisfied with the legal environment). A superior alternative could still be to link more closely to digital government service satisfaction. For instance, embedding satisfaction surveys into digital services so that feedback can be received immediately afterwards and be disaggregated by service used. Such results could be used to highlight both problematic and successful areas, and to provide opportunities for the public administration to share knowledge to improve overall user experiences (OECD, 2020[18]).
The supply of evidence for informed decision-making relies, first and foremost, on the existence of data, its collection and collation, and data sharing and interoperability. The supply of evidence is affected by both organisational and policymakers’ capacities to undertake EIDM. This includes both technical and analytical skills as well as training and resourcing.
Data availability and accessibility
In essence, EIDM relies on information that is analysed to become evidence. The result is that there are a number of necessary technical steps to achieve EIDM. Information first needs to exist, and its existence needs to be known. Second, the information needs to be of sufficient quality for policy analysis. Third, the information needs to be available in an appropriate format.
To embed a data- and evidence-driven culture within the public sector, an overall appreciation and understanding of the data value cycle are needed – particularly from leadership (OECD, 2021[19]). One essential aspect is ensuring that civil servants first consider data use before collecting and supplying it. The data value cycle highlights four areas: 1) collection and generation: 2) storage, security and processing; 3) sharing, curating and publishing; and 4) using and reusing data (Figure 3.2). While some recent efforts have been made in these areas, the Czech Republic faces challenges throughout the data value cycle. These challenges hamper Czech policymakers’ ability to provide evidence to improve decision-making in the country.
The quality and availability of data are crucial for EIDM. Across OECD countries, challenges often relate to data access in the public sector, including understanding the administrative data currently available in ministries. Box 3.2 summarises the types of data sources for decision-making. A broader data challenge is the capacity of the public sector to generate the type of high-quality data necessary to produce evidence for decision-making (OECD, 2020[21]).
Data sources of general applicability
Statistical data: commonly used in research, corresponds to census data or, more generally, to information on a given population collected through national or international surveys.
Administrative data: these data are generally collected through administrative systems managed by government departments or ministries and usually concern whole sets of individuals, communities and businesses concerned by a particular policy. For instance, it includes housing data, tax records and other government data.
Big data: mainly drawn from a variety of sources such as citizen inputs and the private sector, big data are most often digital and continuously generated. It has the advantage of coming in greater volume and variety.
Illustrative specific data sources
Stakeholder data: these data are generated by stakeholders (e.g. businesses, citizens, non-governmental organisations, academia, etc.) and could be for regulatory (e.g. registration, bookkeeping, etc.), accreditation and/or membership (e.g. to apply for government grants, seek membership of an umbrella organisation, etc.), or other reasons (e.g. to provide evidence of survivability before seeking external finance, sharing research, to commence public debate on issues, etc.).
Monitoring data: these data can be generated to assess whether objectives have been met, whether the behaviour of regulated entities has changed as expected or to ensure regulatory compliance, for example. It can be based on field visits, questionnaires or reporting requirements to regulators.
Evaluation data: these data are collected for the purpose of an evaluation. They can take the form of qualitative questionnaires, on-site observations, focus groups or experimental data.
Source: Based Results for All (2017[22]).
Data availability and accessibility are important factors in data use, as data need to exist but also be accessible for analysis. Publishing data is important, as analysts may not otherwise be aware of existing data sets. The OECD OURData Index, which measures the accessibility, usefulness and reusability of public data, shows that the Czech Republic sits very close to the OECD average in all areas (Figure 3.3).
Exploring the data in more detail highlights that the largest area for improvement is in terms of data reuse and, in particular, data promotion initiatives and partnerships. These results support the observations made as part of the review that academic and other institutions generally lack access to data to undertake public policy research. More broadly, the European Commission noted that the current arrangements create poor incentives for collaboration between researchers and business. The European Commission concludes that, more worryingly, there are no proposals for improving the regulatory environment as part of the latest Recovery and Resilience Plan, leaving the necessary reforms to be pursued via domestic reform channels (European Commission, 2022[24]).
In part, the relative lack of open data may stem from an absence of a government-wide approach to data generally. The latest OECD information on member countries’ data policies highlights that the Czech Republic has a relatively fragmented data policy (Figure 3.4). The Digital Government Index assesses and benchmarks digital government policies’ maturity and implementation under a coherent and whole-of-government approach (Figure 3.5).
While all OECD countries have room to improve their digital government practices, some of the countries that score below the Czech Republic overall have superior results relating to the data-driven public sector composite indicator. The data-driven public sector dimension measures the extent to which governments have adopted and implemented a data governance approach to secure the effective management of data across public sector organisations (OECD, 2021[26]). The areas covered highlight some of the current challenges of the Czech public administration and, as such, provide suggestions for reform. From an EIDM perspective, the most germane are the objectives under a public sector data policy that cover matters such as regulatory foresight, evidence-based policymaking, engaging societal stakeholders, developing user-driven services, public sector productivity and efficiency, policy evaluation, monitoring, and organisational learning (Ubaldi and Okubo, 2020[25]).
Data openness is an integral aspect of EIDM. Apart from the clear transparency and challenge benefits that it provides, a wider range of benefits can accrue to governments as a result of data openness (Ubaldi, 2013[28]). In particular, open data has the potential to trigger a revolutionary approach to how governments think about providing services and how they measure efficiency in service delivery and user satisfaction (OECD, 2018[29]). The development of the Open Data Portal serves as a single place for the open data agenda in the Czech Republic. The portal integrated methodological procedures for opening and cataloguing open data for all types of public administration institutions (Ministry of the Interior of the Czech Republic, 2021[30]).
Data sharing and interoperability are important for building awareness of evidence to then use for decision-making. By working in open formats by default, many issues associated with data sharing and interoperability can be greatly reduced. Moreover, it unlocks the potential for data and broader societal innovation. The enhanced ability to combine different data sets can help develop additional, more innovative, and better products and services. In Norway, for example, citizens can lodge development applications via its national online portal, Altinn, which is, in turn, linked to both the relevant municipality and architectural services. Further, Altinn’s peer-to-peer service allows citizens to notify their neighbours of the application for the purposes of public consultation (OECD, 2020[18]). Mixing public data with commercial, civil society and citizen input data, and pooling and sharing with those produced by other public agencies and/or levels of government – i.e. data sharing for developing shared content, services and policies between cities or countries – holds considerable potential for creating public value (OECD, 2018[29]). Data within the Basic Registers have interoperability functionality despite not being based solely within the Ministry of the Interior (see Chapter 5). That said, access restrictions to the data within the Basic Registers means that there is scope for it to be more widely used in EIDM.
The Czech Republic recognised the challenges to both data openness and interoperability in its Strategic Framework for the Development of Public Administration of the Czech Republic 2014-2020 (“Strategic Framework 2014-2020”) (Ministry of the Interior of the Czech Republic, 2014[31]) with the promotion of its open data principles. Addressing the challenges, however, continues to be an ongoing process, despite the fact that the Strategic Framework ended several years ago. The Ministry of the Interior provided a recent update regarding the Implementation Plan of the Strategic Framework, noting some improvements to the quality and interoperability of public administration data (KODI) and the development of the Open Data Portal. It also noted complementary work undertaken to describe the architecture, create rules for publishing data and provide support to public administration institutions in implementing them (Ministry of the Interior of the Czech Republic, 2021[30]).
However, outside of the Basic Registers, one significant challenge faced by the Czech public administration relates to data interoperability. Some published data sets only include aggregate data that cannot be merged or linked with other data sets, and thus are of little use for statistical analysis. Beyond aggregate data, stakeholders interviewed for this review repeatedly mentioned that accessing micro-level information from the Czech Statistical Office is difficult. The difficulty stems from the fact that agencies must go through various procedures each time they want to obtain unpublished data. Examples from other OECD countries, such as Denmark (Box 3.3), suggest that greater data availability does not have to be at the expense of privacy or cost concerns in its potential for use. Building on open data principles, over time, the Czech Statistical Office, in co-operation with the various agencies, could investigate the possibility of publicly releasing requested data to enable other interested parties to undertake their own analysis. An additional benefit from such an approach could be a reduction in the resources needed to address repeat data requests received from various parties.
In Denmark, personal data are stored in registries with personal identification numbers. Statistics Denmark facilitates the use of these micro-level databases for research purposes for approved analysts, universities, research organisations and ministries. Statistics Denmark possesses data in more than 250 subject areas ranging from labour markets, consumption and demographics to transport, agriculture and the environment. The data are prepared by the Research Service Division and are accessible remotely and securely through specific Internet servers. Analysts can access data in these areas as far back as the 1970s.
Source: Statistics Denmark (2014[32]).
Another barrier to data use in EIDM is privacy. All OECD countries face the challenge of balancing the use of personal data for EIDM and ensuring that the personal data rights of citizens are secured and respected (OECD, 2020[33]). Indeed, data protection legislation can constitute an obstacle to using individual-level data for decision-making in some countries, specifically when carrying out statistical analysis and merging files, which requires access to single identifiers (OECD, 2020[33]).
Stakeholders interviewed for this review frequently cited privacy concerns as a central constraint to a more open and transparent data culture across the Czech administration. While privacy should be recognised as a legitimate interest when weighing concerns about the public release or use of specific data, it should not operate as an effective veto on providing information for decision-making. The existence of the Basic Registers points to the fact that the Czech administration can balance privacy concerns with data access.
Research institutions have the ability to critically assess evidence presented by the government and can undertake research independently of the government. In turn, governments can rely on such research as part of EIDM, with the additional advantage of saving public resources, as the work has already been carried out. A corollary to the lack of publicly available – and especially linked – data means that Czech research institutions cannot act as an important voice as part of EIDM. One possible solution could be for the Czech government to create formalised agreements with research institutions covering matters of data access and use. In Denmark, for instance, there are agreements between all research and analysis institutions and Statistics Denmark, which, among other things, clarify roles and responsibilities and ensures that the staff of research and analysis institutions handle their administration of each institution in the most appropriate way (Statistics Denmark, n.d.[34]).
More generally, systematic strategies and policies to combine, link and reuse data, as well as to connect actors and decisions within and outside the public sector, are necessary to enable administrative data to be used for EIDM (OECD, 2019[35]). As a result, some OECD countries have sought to develop EIDM strategies by fostering systematic use of administrative data (Box 3.4). Such a government-wide strategy for using administrative data in decision-making could be included in a broader framework on EIDM in the Czech Republic.
The United States has institutionalised and implemented government-wide approaches for using data for analysis. It has done so by mobilising institutional resources, promoting internal champions and exploring the possibility of fully using existing data systematically through significant governance changes. The United States issued the 10-year Federal Data Strategy centred around three core principles: ethical governance, conscious design and a learning culture. The Federal Data Strategy is complemented by an implementation plan of 40 practices to help agencies comply. Moreover, the Foundations for Evidence-Based Policymaking Act of 2018 includes a government-wide approach to data as a key pillar for the evidence-informed policymaking vision. Its implementation plan mandates agencies in the US administration to have a chief data officer. It also covers such programmes as “Open Data Access and Management” and “Data Access for Statistical Purposes”.
Sources: OECD (2019[35]; 2021[19]).
As open data maturity increases across the globe, so is governments’ understanding that securing long-term sustainability requires changing the organisational culture to further the leadership’s understanding and build the necessary skills and open-by-default mind set among civil servants and across the public sector. This can indeed facilitate the scaling up of successful initiatives, the sharing of knowledge on what works and what doesn’t, and the promotion of open data reuse by civil servants, who can become active agents in benefits realisation (OECD, 2018[29]). It is also central to embedding an evidence-based culture more broadly within the civil service to engage in policy experimentation and monitoring and evaluation.
Civil servants are a key group of potential data reusers. Fostering a culture that sustains the use of data to innovate “business processes” and create collaboration within the public sector is essential for long-term sustainable results. This implies building capacity to reuse data for decision-making, in particular, related to strategic foresight, innovation (e.g. building data analytics capacities, see below) and improving performance (e.g. releasing data and implementing open data policies considered essential as part of the performance indicators framework or policy) (OECD, 2018[29]).
Data sharing and interoperability appear to be at the heart of data issues in the Czech Republic (see above). One issue relates to the digitalisation of government information (see Chapter 5). However, more fundamental data issues remain, especially with regard to data culture within the Czech administration. Stakeholders of this review noted that data accessibility was an acute issue, and it was recently noted by Deputy Minister for Science, Research and Innovation Štěpán Jurajda, that:
We actually have most of the data we need. But they are not accessible to researchers and do not connect across departments, because each of them has its own information system. It is easier for each report to say that it will not give the data to anyone because it does not collect it for research purposes or because it does not have a way to anonymise and protect the data.
The main reason for this state of affairs is the fact that we do not have legislation in place to regulate access to public data as is common in developed countries. As a result, the necessary data are not available at all, and when someone does get access to it somewhere, it is a specific, sometimes random access. So it can’t work well. (Hrstková, 2022[36])
Data stewardship was also identified as an issue. In the Czech Republic, many civil servants effectively have sole discretion about whether to release data, to whom and the conditions for using it. Many other countries have permanent entities responsible for granting access to data. Not only does this allow for administrative savings where the same data are repeatedly requested by different institutions, but it also helps to ensure that data requests are treated consistently across the entire administration. On this latter point, an additional issue is that while some informal networks and channels do exist to help facilitate some data sharing, these are solely dependent on the personnel working there. If they change positions, these links are broken and both the corporate memory and access to the data are potentially lost.
The Ministry of the Interior’s current proposal on public data management recognises problems with the current system. Apart from giving due consideration to the factors identified here, implementing the final proposal (Box 3.5) will be critical to bringing about the necessary cultural change needed to ensure that evidence is both more widely available and used in decision-making.
The proposal on public data management aims to introduce the principles of public sector data management and regulate the controlled access to public sector data and their reuse.
The first part of the draft proposal stipulates that mandatory entities are obliged to carry out a data review, during which the data held by the given mandatory entity will be identified. Based on the data review, descriptions of these data will then be created and recorded in the public register. The public administration and the public will thus know what data the state has and works with. Obliged entities will also formalise and guarantee the accessibility of identified data both for the performance of their agendas and the agendas of other obliged entities.
The second part of the proposal is the partial implementation of the so-called DGA regulation. The proposal creates a legal framework for a controlled data access mechanism. Under clearly defined conditions, this will enable access and reuse of data that are currently excluded from the open data by default principle. The proposal contains a system of tools that ensures the protection of conflicting rights and interests, especially in the form of the right to the protection of personal data.
The proposal further envisages the establishment of a single data office that will act as a central platform for processing requests for controlled access to data and connect the data interfaces of different entities.
Source: Information provided by the Ministry of the Interior, 2022.
Analytical capacities to provide evidence
The OECD has recently published reports on the supply side of EIDM on mobilising evidence and building capacities (OECD, 2020[33]; 2020[1]). Key factors relate to organisational and policymakers’ capacities to provide EIDM.
Organisational capacity
Organisational capacity refers to factors that can either support or impede the use of evidence within organisations. This can include tangible factors such as well-maintained computer facilities, adequately resourced libraries and robust knowledge management processes. Evidence cannot be disseminated or translated if such resources are unavailable or cannot be accessed in time (OECD, 2020[1]).
Key aspects of organisational capacity are culture, inertia and political context. Culture refers to the norms, values and basic assumptions of a given organisation (Damschroder et al., 2009[37]). Inertia refers to potential resistance faced in instituting change management (Godkin and Allcorn, 2008[38]). Political context here refers to the civil service delivering on potential appetite for reform, which is a demand factor for EIDM previously discussed.
Decisions about how to gather, analyse and interpret evidence will also be shaped by the internal dynamics of individual government departments, as well as the wider bureaucratic and political pressures (Shaxson, 2019[39]). This includes civil service reform programmes, organisational cultures, and internal structures and processes that impact how individuals and teams work with each other. Cultural and attitudinal factors in the wider society also affect the extent to which evidence gets used in policymaking (OECD, 2020[1]).
The Czech administration chronically lacks analytical capacities. It is very rare for ministries to have an analytical and/or statistical unit.2 When they do, these units rarely contribute to developing RIAs or other cross-cutting policy documents using evidence-based methods. The example of RIA can illustrate this.3 Research of the Government Office has shown that, instead of multidisciplinary teams, with some exceptions, there is only one person responsible for drafting an RIA (Office of the Government of the Czech Republic, 2020[40]) and this is very often a lawyer from a legislative department. There are some research institutes that report to some ministries (e.g. the Research Institute for Labour and Social Affairs, the Institute for Occupational Safety Education, and the Occupational Safety Research Institute, all of them reporting to the Ministry of Labour and Social Affairs); however, according to interviews conducted for this review, all of these research institutes contribute to the analysis needed for RIA or policymaking.
The current government has decided to tackle the lack of analytical capacities by creating a Government Analytical Unit (VAU) in the Office of the Government. This Unit should focus on providing expert analysis of horizontal issues and assist individual ministries in carrying out economic analysis when developing new policies and/or laws. In the initial stage, VAU experts should assist in drafting individual RIAs if asked for advice, but RIA should not be the sole focus of the analytical work of the VAU in the long run. Eventually, analytical support of strategies and policies, as well as strategic foresight, should be added to its responsibilities. Additional tasks could also include support activities, such as networking with research institutes within and outside the administration, universities and think tanks; piloting and testing new policies and regulations; co-ordinating the introduction of behavioural insights in the administration; and/or other activities aiming at better EIDM development across the Czech government.
The VAU has been set up in the Office of the Government under the responsibility of the Minister for Legislation. Creating the VAU was an ad hoc initiative of the new government. This has also attracted some criticism in terms of “adding more bureaucrats to fight bureaucracy”, which might stem from misunderstandings regarding the purpose of the VAU. The VAU could be used as a basis and an important first step in strengthening analytical capacities in the administration. In the long term, similar departmental analytical units should be created with cross-cutting support provided by the VAU. This should also lead to transferring some of the central VAU staff to these departmental units.
The creation of the VAU is mostly in line with the objective set by the Ministry of the Interior in the National Recovery Plan to be launched in 2023, focusing on strengthening analytical capacities in the public administration. Its aim is to create a position of analytical co-ordinator in each ministry and, through these co-ordinators, train and methodologically support analysts in respective ministries. Co-operation among these co-ordinators should be ensured through an inter-ministerial working group. At the same time, a new Central Analytical Unit should be created which will increase awareness on the importance of EIDM and will provide guidance and methodological support to individual ministries and public administration agencies on qualitative and quantitative analytical methods. However, this project is supposed to be carried out under the leadership of the Ministry of the Interior. It would be important to better co-ordinate such activities in the future to achieve tangible results and avoid tensions relating to which body is the leading one. The VAU should be used as a basis through which analytical capacities can be strengthened across the administration, and the existing political momentum should be exploited in this regard.
Policymakers’ EIDM capacities
Increasing the prevalence of EIDM by policymakers depends on behavioural change, such as using evidence and evaluation to influence policy debates, the resulting policy choices and the practical implementation of those choices (Langer, Tripney and Gough, 2016[41]). This can be conceptualised as components in an interacting system. A model developed by Michie, van Stralen and West (2011[42]) posits that capability, motivation and opportunity interact to generate behaviour:
Capability is defined as an individual’s psychological and physical capacity to engage in a specific activity. It includes having the necessary knowledge and skills.
Motivation is defined as all the processes that energise and direct behaviour, not just goals and conscious decision-making. It includes habitual processes, emotional responding and analytical decision-making.
Opportunity is defined as all the factors outside the individual that make the behaviour possible or prompt it.
Capability to engage in EIDM includes an individual civil servant’s knowledge of different types of research methods, as well as fundamental skills of statistical and data literacy and the capacity to read and understand analytical products, often in English. Box 3.6 provides the requisite skill set for civil servants to undertake EIDM. Motivation to engage in EIDM can include beliefs that civil servants have a mandate to use evidence, that the use of evidence will be rewarded, and an understanding of how the use of evidence will improve the quality of policymaking and ultimately increase trust in government. The opportunity to engage in EIDM includes the strength of the connections between the policymaking and the research community and civil servants’ institutional access to evidence (OECD, 2020[1]).
Research collaboration between the OECD and the Joint Research Centre of the European Commission led to a definition of the requisite skill set for evidence-informed policymaking to comprise the following elements:
Understanding evidence-informed policymaking – Policymakers with this skill will understand the role of evidence and its place in the policymaking cycle as well as the challenges and opportunities which come with the use of evidence. This will be underpinned by knowledge of different research methods and their purpose, as well as the fundamentals of statistical and data literacy (including big data, machine learning and artificial intelligence).
Obtaining evidence – Policymakers with this skill will be able to gather existing evidence in their own policy area and know who to turn to as sources of evidence synthesis. They will be able to recognise where there are evidence gaps and commission high-quality evidence to fill the gaps using a range of project management techniques.
Interrogating and assessing evidence – Policymakers with this skill will make use of a set of holistic, systemic and critical thinking tools. They will be able to assess the provenance, reliability and appropriateness of evidence. They will have the ability to interrogate evidence by critically assessing its quality and context using a range of techniques to challenge assumptions and biases.
Using and applying evidence in policymaking – Policymakers with this skill will understand their own policy context and recognise possible uses of evidence in the policy cycle. They will be proficient in knowledge management and understand the role of innovation, with an ability to assess and manage risks and challenges. They will be familiar with and know when to use innovative techniques like behavioural insights, design thinking, policy labs and foresight to support policy design and implementation.
Engaging with stakeholders in evidence-informed policymaking – Policymakers with this skill will have strong engagement and communication skills, including the ability to create effective evidence-based messages for different types of audiences and to engage and inspire a variety of stakeholders. They will be able to manage and facilitate evidence-informed debate with policymakers and citizens and maintain collaboration with the evidence community. They will have a good grasp of co-creation, co-production and participatory methodologies.
Evaluating the success of evidence-informed policymaking – Policymakers with this skill will understand different evaluation approaches and tools and know how to use comparative examples to inform evidence-informed decision-making (EIDM). They will understand that evaluation should be built into the policy cycle and should serve to inform and improve EIDM. They will know and use qualitative and quantitative indicators of successful evidence use.
Policymakers with these skills will be better placed to conduct EIDM as part of their daily work.
Source: OECD (2020[1]).
Policymakers’ analytical skills encompass an ability to both create and use information in decision-making. This includes an individual’s knowledge of different types of research methods as well as statistical and data literacy, and the capacity to read, understand and utilise analytical outputs. In practice, these skills often require a multidisciplinary set of competences drawing from a wide range of areas, including economics, statistics, social sciences, environmental sciences, law and engineering.
The main challenges for attaining these capacities according to the various participants are:
a severe lack of analytically skilled staff, due in part to a lack of suitably qualified staff in the Czech labour market and difficulty in attracting and retaining such staff
the scarce analytical skills that do exist are dispersed in an ad hoc way across ministries
a general lack of training and analytical capacity building, which perpetuates the notion that skills are neither in demand nor desired across the Czech public administration.
One additional challenge is decision-makers’ general lack of demand for evidence. This, in turn, influences the skills that senior policymakers seek when recruiting (see section above). A solution undertaken in Lithuania was to create a whole-of-government evidence and analysis unit (Box 3.7).
In 2017, the Lithuanian Science and Education Monitoring and Analysis Centre (MOSTA), previously located at the Ministry of Education, Science and Sport, was moved under the responsibility of the Office of the Government. This transfer was motivated by a strategic need for leadership in the generation of evidence and analysis for the whole of government. In 2019, MOSTA was officially transformed into the Government Strategic Analysis Centre (STRATA) with the mission to foster high-quality evidence and public knowledge based on objective information. The intention is to leverage the expertise of the centre to strengthen the evidence-informed decision-making mechanisms to enable sound strategic governance from a whole of government perspective.
The current legal framework gives STRATA a wide variety of responsibilities, which intervene at different stages of the policymaking cycle. Indeed, the Government Strategic Analysis Centre is responsible for:
carrying out foresight activities, monitoring and evaluation in the context of the strategic governance system
conducting thematic studies in the areas of expertise related to its previous MOSTA mandate
promoting the quality of regulatory impact assessment and ex post assessments
providing advice to promote evidence-informed decision-making
STRATA has been given a mandate in several stages of the preparation and implementation of the main strategic planning documents of the Lithuanian government. In particular, STRATA has an explicit role in conducting strategic foresight for the preparation of the State Progress Strategy 2050 and the National Progress Plan 2030, as well as monitoring and evaluating these plans.
STRATA has a role in promoting the quality of regulatory impact assessments (RIAs) by providing quality control for high-impact RIAs and general methodological support to ministries. Since 2020, quality control of higher impact legislation has been delegated to STRATA. STRATA, together with the Office of the Government, reviews the preliminary information on RIAs sent by the ministries and decides which legislative projects should be included on the semi-annual high-impact legislation list. In 2021, these semi-annual lists were substituted by a list covering a three-year period (2021-24) with the possibility for revision. Once the list is completed, the ministries drafting these legal acts can solicit methodological help from STRATA. The Office of the Government sends the final RIA to STRATA for quality control. STRATA controls the quality of the impact assessment.
STRATA also has a role in quality assurance, by offering support to ministries that are drafting “proposals of evidence-informed decisions”, which includes RIAs. In this regard, STRATA has co-operated with ministries on conducting RIAs. One such example is the ex ante impact assessment of the COVID-19 relief stimulus, where STRATA provided its expert opinion on the impact estimated by the Ministry of Finance. Furthermore, STRATA developed some cross-government RIA methodological guidelines.
Furthermore, STRATA may also have the mandate to provide quality assurance for ex post regulatory assessments upon request. The Ministry of Justice is in charge of co-ordinating ex post regulatory assessments and plans to solicit STRATA’s advice on the quality of ex post assessments.
Source: OECD (2021[19]).
Analytical expertise is not co-ordinated
Stakeholders interviewed for this review noted that there are pockets of analytical expertise scattered across a number of ministries. However, the staff’s expertise was more a result of their own volition rather than a top-down edict from decision-makers or senior policymakers that these skills are desired. Further, in practice, the extent to which analytical expertise could be deployed is largely self-determined by the relevant staff member who has the skills, if they are afforded the time and have the desire to do the work.
The general lack of a co-ordinated approach to analytical expertise represents a missed opportunity for better evidence-informed decision-making in the Czech Republic. According to the Ministry of the Interior’s plans, in the future, co-ordination among analytical units should be ensured by the central analytical unit and its network of ministerial analytical units. Solutions in other OECD countries have looked to facilitate a central co-ordination role to establish whether policies are working as intended (Box 3.8). The VAU could play this role in the future.
The United Kingdom’s What Works initiative aims to improve the way government and other organisations create, share and use (or “generate, translate and adopt”) high-quality evidence for decision-making. It supports more effective and efficient services across the public sector at national and local levels. The What Works Network is made up of seven independent What Works centres and four affiliate members.
A What Works national adviser in the Cabinet Office promotes and supports the independent What Works Network and carries out the following cross-cutting initiatives:
running a Cross-Government Trial Advice Panel, with experts from across academia and government providing a free service for all civil servants to help test whether policies and programmes are working
sharing findings from the What Works centres across government and promoting discussion on “what works”
supporting the development of a civil service with the skills, capability and commitment to use evidence effectively
helping policymakers take informed judgements on investment in services that lead to impact and value for money for citizens.
Source: UK Government (2022[43]).
Analytical skills are scarce in the Czech labour market
One significant issue in the Czech public administration is that its recruitment policy requires previous civil service employment and is preferred to other experiences in the private or community sectors (see Chapter 6). At an organisational level, this poses the risk that the public administration is not generally benefiting from external knowledge in managerial positions, especially those who can bring their own experiences about driving behavioural and cultural change.
Analytical skills are crucial to ensure the effective supply and use of evidence for decision-making. In particular, quantitative skills, data skills and related soft skills are extremely important in a world that is becoming ever more digitalised. The volume, velocity and variety of data have increased dramatically and “data literacy” among civil servants is indispensable (OECD, 2017[44]). Data scientists or economists/statisticians competent in working with data must be present among ministerial staff so that the evidence derived from data is used correctly, and that external evaluations and assessments are contracted appropriately. This might require developing more programmes focused on quantitative analytical skills, particularly economic skills.
The Czech Republic has a relatively low share of adult learning among public sector workers (10.2% in 2021 compared to the EU27 average of 18.6%, likely aggravated by the COVID-19 pandemic restrictions). The Czech Republic also remains below the EU average in its share of public administration employees with a tertiary education (45.3% compared to the EU average of 55.3%). The EU-level target for tertiary education attainment remains distant. The Czech Republic also faces challenges in attracting young civil servants. It ranks in the bottom half of the EU on the share of government employees under 39, leading to potential concerns about the stability of the civil service (European Commission, 2022[24]).
Training and capacity building is insufficient for EIDM to take hold
Training is a cornerstone of EIDM, as it is unlikely that all civil servants recruited will have the complete skill set to undertake EIDM immediately. As noted above, while data gaps do exist, issues around data sharing and an acute lack of data interoperability across ministries’ systems are problematic. Consequently, it is more likely that there will be a lower level of understanding of EIDM as, for example, undertaking anonymised linked data research is seldom currently required.
The Czech administration has recognised that ongoing training for civil servants is required in a number of areas, including those relating to EIDM. The Czech Ministry of the Interior and its Department of Chief Architect of E-Government regularly organise open data training courses and workshops for public servants. These training courses aim to equip public servants with the adequate knowledge and skills for publishing government data as open data. These training courses thus also include open data literacy skills development (OECD, 2018[29]). Notwithstanding the volume of training undertaken – more than 1 000 civil servants undertook the training courses in 2021 – the extent to which this has embedded a more open and data-centric approach across the public administration is unclear. Metrics used as part of the Ministry of the Interior’s annual reporting cover the number of open data providers and the number of data sets available (see below), but usage indicators are missing. The Evaluation Unit within the Ministry for Regional Development commenced an annual evaluation conference series in 2015 that looks mainly at the evaluation of European Structural Fund programmes. Through the open discussion of evaluation techniques and experience sharing, the conferences provide an opportunity to bring evaluation practitioners together to discuss what works and what does not in the field of evaluations. Conferences have included enhancing the clarity of communicating results, improving the impact of evaluations, and better linking evaluations to management and policymaking (Ministry for Regional Development of the Czech Republic, 2015[45]).
Nevertheless, a severe lack of policy evaluation and analytical capacities within the civil service was identified as a significant hurdle to overcome if EIDM is to flourish in the Czech Republic. The OECD Digital Government Index found few training initiatives for public professionals in areas such as data analytics in policymaking and service delivery (8 countries or 28%), artificial intelligence (9 countries), and usability and accessibility (6 countries each). Examples of such a comprehensive training approach for the public workforce are the GDS Academy in the United Kingdom and the School of Public Service in Canada (OECD, 2021[46]).
Both the RIA Board and the RIA Unit within the Office of the Government (see below) provide training and capacity building for civil servants in the application of RIA (OECD, 2021[7]). The OECD received examples where individual staff from several different ministries had undertaken RIA on their own initiative to help inform policies for decision-makers. However rare, these instances demonstrate that current policymakers can undertake RIA in the Czech administration. A significant challenge signalled was that even in these instances, RIA tended to start very late in the decision-making process. Starting late meant it was more difficult to obtain information on potential impacts and engage with affected parties. It should also be noted that while the exact functions of the VAU are not yet defined, it is expected that one of its roles will be to provide training to civil servants.
A lack of resources can be a barrier to undertaking EIDM. Resources relate not only to ensuring that future data costs are budgeted for, but that significant investment is made in carrying out all aspects of EIDM. This means adequately resourcing ministries to conduct evidence-based policymaking, as well as having appropriate oversight resources. The staff resourcing in the RIA Unit is comparable to the OECD average, although it should be noted that resourcing levels vary considerably across countries (OECD, 2021[7]).
Assessment of key EIDM institutions and documents
Various government institutions are responsible for generating demand for EIDM in the Czech Republic. The main government entities are the Ministry of the Interior, which is responsible for the overall public administration, including the civil service, and the Office of the Government. These entities are responsible for generating various government documents such as strategies and guidance relating to, among others, better regulation. Government councils play an important role in both assessing and fostering public scrutiny of regulatory proposals (see Chapters 1 and 2 for a discussion of government councils).
The final major institution relates to regulatory oversight. The RIA Board was constituted in 2011 and is responsible for scrutinising regulatory proposals of the government subjected to RIA. More recently, the Czech government announced the establishment of the VAU in the Office of the Government, whose final role is currently unclear.
Assessment of key government documents
Key documents include the Competency Law; the Government Legislative Rules of 1998, were updated in 2018; various government strategies; and better regulation guidance. Like all OECD countries, the Czech Republic has a published government-wide regulatory reform policy (OECD, 2021[7]).
Chapter 2 discusses the Competency Law in more detail. The Government Legislative Rules set out the general rule-making procedures in the Czech Republic. Government strategies relevant to better regulation, as well as bespoke guidance, are discussed below.
The overarching government strategies relating to EIDM are the:
Strategic Framework 2014-2020 (Ministry of the Interior of the Czech Republic, 2014[31])
2030 Strategic Framework for the Czech Republic (Government of the Czech Republic, 2017[14])
Methodology for the Preparation of Public Strategies (Ministry for Regional Development of the Czech Republic, 2018[47])
Client-oriented Public Administration 2030 (Ministry of the Interior of the Czech Republic, 2019[48]).
It was previously recognised that the Czech public administration lags behind its European peers in a number of areas (Ministry of the Interior of the Czech Republic, 2014[31]). The lack of a conceptual document for the development of public administration was noted as part of the rationale for the Strategic Framework 2014-2020. The other part was in response to pre-conditions imposed by the European Commission in allowing for the drawdown of funds under the European Structural and Investment Funds (Ministry of the Interior of the Czech Republic, 2014[31]). The last point is particularly pertinent to EIDM, as the European Structural and Investment Funds imposed strict monitoring and evaluation requirements which, in turn, provided for various information and data needs to track progress.
The Strategic Framework 2014-2020 recognised that the regulatory environment in the Czech Republic could be improved. Several of its strategic objectives related to improving EIDM. For example, Strategic Objective 1.2 aimed to improve RIA, develop a methodology to assess administrative burdens and create an ex post RIA mechanism (Ministry of the Interior of the Czech Republic, 2014[31]). According to the evaluation report, the Ministry of Trade implemented 62 measures to reduce administrative burdens on businesses in 2015, which exceeded the original goal of 60 measures. There were 40 additional measures identified and most of them were implemented by 2019. In the meantime, the Ministry of the Interior started implementing a project to reduce administrative burdens for citizens and the public administration.
The completion and implementation of the proposed outputs of various strategic documents consistently remain a weak point in the Czech administration. An important goal of the strategy was developing a framework for regular ex post performance evaluation of existing regulation through an ex post RIA. A methodological framework was developed by two working groups under the leadership of the Office of the Government. However, this proposal was rejected by the Government in 2021 and is still being finalised in the inter-ministerial comment procedure and its destiny is not clear.
A previous evaluation of the Strategic Framework noted that elements of process management and the introduction of selected aspects were delayed. Among others, specific objectives relating to reducing administrative burdens and the introduction of the measurement and evaluation system were not completed on time (Ministry of the Interior of the Czech Republic, 2017[49]). Moreover, it was noted that promoting the particular needs of special interest groups or of politicians runs counter to EIDM and makes it difficult to implement the Strategic Framework as originally intended (Ministry of the Interior of the Czech Republic, 2017[49]).
While the Strategic Framework Czech Republic 2030 relates to sustainable development, it touches on a number of EIDM aspects. In particular, it includes a chapter on good governance that notes: “Citizens participate in decision-making on public affairs and the state creates suitable conditions to facilitate this. Public administration enhances the quality of life of the population of the Czech Republic via public policies and achieves the goals of sustainable development in the long-term perspective” (Government of the Czech Republic, 2017[14]). One specific objective is to “increase the inclusiveness of governance, i.e. the real possibility of involving its citizens in decision-making on public affairs, or more precisely into the policymaking process” (Government of the Czech Republic, 2017[14]). The means that achieving that objective will be through “greater use of participatory forms of democracy which emphasise direct involvement in the decision-making process, and deliberative democracy oriented towards discussion and mutual persuasion. For example, public consultation on legislative proposals, whether by the government, or by anyone else, using referendums or inviting citizens to co-decide on budgetary priorities (participatory budgeting), especially at the local level” (Government of the Czech Republic, 2017[14]).
Improving the evidence base is also an objective of the Strategic Framework: “Another important step towards achieving policy coherence will be to strengthen the strategic approach and [place a] greater emphasis on preliminary impact assessments and retrospective evaluation. For each decision, consideration should be given to the impact on other areas (side-effects), especially long-term impacts and, if possible, to developing alternative solutions” (Government of the Czech Republic, 2017[14]).
The strategic objectives under the good governance key area most pertinent to EIDM are that policymakers have high-quality and easily accessible data and information for decision-making needs, innovative solutions are fostered to improve long-term policy effectiveness, and user satisfaction of public administration increases (Department of Sustainable Development, 2021[50]).
The 2030 Strategic Framework for the Czech Republic included a typology of tools to contribute to attaining its objectives. In total, the 5 strategic goals comprised 15 specific goals and 59 measures and recommendations. A review of the Strategic Framework for Sustainable Development highlighted both the concentration of favoured tools and the relative neglect of others. Closing off on the use of a broad range of tools can unduly limit the evidence base for decision-makers. A narrow evidence base presents decision-makers with few realistic options among which to choose (OECD, 2021[5]). The review highlighted that nearly 60% of the selected tools relate to strategic/planning/project and organisational institutional/process. While in and of itself this may not be a problem, the review highlighted some shortcomings with the relative marginalisation of the use of other tools:
An example can be Objective 24.2, intended to advance the ex ante evaluation of the impacts of public policies. The latter is solved only by tasks of a strategic and planning nature. If we broaden the scope to the entire strategic objective, the representation of other types of instruments will increase; however, within the framework of a specific objective, it is essentially a “bet on one card”... For each specific goal, it would therefore be necessary to rethink whether it makes sense to concentrate on one type of tool (because they are the only one possible, because they are by far the most effective, because they are the only ones still lacking in the policy mix for the given issue, etc.), or if it is just the fact that there is no will, capacity, etc. to use other types of tools [original emphasis]. (Department of Sustainable Development, 2021[50])
It was particularly noted that participatory and deliberative tools were underutilised. For instance, in attaining the key objective of resilient ecosystems, it was noted: “The practical absence of participatory and deliberative tools shows a strong fulfilment of the assumption of using only expert knowledge and capacities for the strategic management of areas falling under the key area of resilient ecosystems. And thus a possible significant abstraction from the discussion of the given issues with the public or scant efforts to involve the public in decision-making processes” (Department of Sustainable Development, 2021[50]).
The Methodology for the Preparation of Public Strategies commenced in 2018. Along with complementary documents referenced in the methodology, such as the methodology of a systematic overview of knowledge for the creation and evaluation of public strategies and A Guide to the Project Management Body of Knowledge, coupled with the Typology of Strategic and Implementing Documents (Ministry for Regional Development of the Czech Republic, 2017[51]), they provide an overall framework to devise public strategies in the Czech Republic.
Given their ubiquity – there are now nearly enough government strategies to cover every day of the year – getting their design, implementation and review right is of crucial importance for the functioning of the Czech public administration. Yet, a fundamental problem is getting the Czech public administration to use the methodology when creating public strategies. A previous evaluation of the Strategic Framework highlighted that less than one-quarter of public strategies were prepared according to the methodology (Ministry of the Interior of the Czech Republic, 2017[49]). Designing public strategies without much degree of commonality makes reconciling differences more difficult and leads to a more complex policy framework. This, in turn, makes monitoring and evaluations more isolated and limits the possibilities for horizontal learning.
There is recognition of the crucial role that EIDM plays in the creation of strategy documents (Box 3.9). The material, therefore, provides a sound basis for policymakers to devise strategies with EIDM in mind. In practice, however, public strategies lack the obligation to provide information to turn into evidence to aid in decision-making. Additional weaknesses identified during the review include a general lack of strategy evaluation, i.e. a “set and forget” mentality pervades; and that there is little co-ordination and coherence in the production of strategies (see Chapter 2).
The guidance material provides that strategies should be prepared:
transparently and objectively, and based on a wide range of interested parties, leveraging existing knowledge wherever possible
in a co-ordinated manner, with clear roles and responsibilities
based on evidence, include predicted impacts, and be later evaluated to determine their real benefit and impact
with an emphasis on outputs that includes specific and targeted measures, including the establishment of metrics to determine success and progress in implementation
to allow for continuous evaluation and experimental learning
with adequate financing to ensure that they can be fully implemented and evaluated.
Source: Ministry for Regional Development of the Czech Republic (2018[47]).
The Client-oriented Public Administration 2030 is the follow-up to the Strategic Framework, which expired in 2020. The Concept was informed by the Strategic Framework and other key documents outlined above, as well as the White Paper on the Future of Europe. A working group was created in 2018 to help design the Concept. It was comprised of key government stakeholders and met five times. A public questionnaire was available for two months in 2018 (Ministry of the Interior of the Czech Republic, 2019[48]). This all suggests that a detailed and transparent process was followed to create the Concept, although how the feedback that was received was incorporated into the final design is unclear.
Better use of EIDM is the first objective of the third pillar of the Concept. The goal is to be reached through:
supporting the creation of analytical teams in the state administration
supporting decision-making based on data and facts and their critical evaluation
publishing analyses and accompanying data in one place to make sharing them easier.
The document also lists increased support for RIA reporting as one of its objectives.
It will be crucially important to ensure that the Concept is appropriately implemented. As noted in the Concept, it (along with the various action plans) is a living document – even more so given their time frame – and there ought to be a degree of flexibility in their eventual coverage. To this end, 12 monitoring and evaluation reports are required to assess the implementation of the Concept and its action plans. As part of its implementation, the Concept requires an ex post impact evaluation of the Strategic Framework 2014-2020 on the Czech public administration. The evaluation should help form a baseline about the level of EIDM that exists within the public administration – for example, as presented in the results of the Ministry of the Interior’s annual report series – that can then be used as a basis to determine the value added of the Concept as it relates to EIDM. It is hoped that these reports are not used as the basis for compliance reporting, however important that may be, but rather as a health check on the implementation of the Concept, unearthing previously unseen potential blind spots and ensuring the continued co-operation and support of all necessary stakeholders.
Assessment of better regulation guidance
As stated above, regulatory management tools, including regulatory impact assessment and ex post assessment of regulations, play a strong complementary role to EIDM. RIA is one of the most important tools for ensuring that policies and laws are based on the best available evidence. The current ex ante decision-making guidance in the Czech Republic dates from 2016 and covers both stakeholder engagement and RIA (OECD, 2021[7]). The obligation to carry out RIA is set by the Legislative Rues of the Government (Government of the Czech Republic, 1998[52]). Detailed guidance is then provided by the General Guidance for Regulatory Impact Assessment.
RIA was introduced to the legislation-making process in the Czech Republic in 2007, after two years of pilot testing. The General Guidance for Regulatory Impact Assessment (Obecné zásady pro hodnocení dopadů regulace) was adopted at the same time. At that time, the Ministry of the Interior was responsible for co-ordinating the RIA framework and overseeing the quality of RIAs.
The RIA framework in the Czech Republic was substantively reformed in 2011. The co-ordination competence was transferred to the Office of the Government and a specific body was created to oversee the quality of RIA – the RIA Board (see below).
According to the Legislative Rules of the Government, ministries and other state administration bodies drafting legislation are obliged to carry out impact assessments following the RIA Guidelines. Impact assessments are required for every legislative draft prepared by the government as well as for any secondary regulations with expected substantive costs to businesses, administrations and citizens. Legislation proposed by members of parliament and local and regional authorities is exempted from the obligation to carry out RIA.
The obligation to carry out a RIA is confirmed at the time of the approval of the Legislative Plan of the Government (or a plan of preparation of secondary legislation). Based on the criteria in the Legislative Rules of the Government described above, the drafting authority indicates whether an RIA will be carried out with the draft (i.e. where significant impacts are expected). The RIA Board has an opportunity to comment on the Legislative Plan and indicate cases where its view on the necessity to carry out an RIA differs from that of the drafting authority.
According to the Legislative Rules, for all legislative documents, an overview of impacts should be carried out before a decision is taken on whether the problem at hand can be resolved through regulation. The overview of impacts contains a description of the problem, the objectives of government action, the identification of affected subjects, a qualitative description of potential impacts and a justification that a legislative solution is needed based on a comparison of various alternatives (this, in fact, constitutes a “light RIA” of sorts). Based on the overview of impacts, it is then decided whether a full RIA will be carried out.
According to the RIA Guidelines, RIA should be carried out whenever new, significant impacts on the following are preliminarily identified in the overview of impacts:
The guidelines do not specify what is meant by “significant impacts”, which is in line with practice in many other OECD countries. It is left to the discretion of individual ministries to decide what constitutes such impacts. This is why the second opinion and control by the RIA Board at the drafting stage of the Legislative Plan of the Government is crucial in identifying relevant legislative drafts suitable for RIA at an early stage.
Any legislative drafts not included in the Legislative Plan of the Government (or the Plan of Secondary Regulations) must be accompanied by a RIA unless the Chair of the Legislative Council grants an exemption from the obligation to conduct RIA.
RIA is not developed in cases of a Constitutional law, the Law on Budget, in cases of a crisis situation and some other cases where impacts are insignificant or the way of implementation has already been set (e.g. by a higher level legal document or an EU Regulation). The Chair of the Legislative Council might grant an exemption from the obligation to conduct RIA only in exceptional cases. However, the number of exemptions seems to be rather high [113 out of 386 proposals in 2020 (Eršil, 2018[53])]. In most cases, ministries justify the exemptions for reasons of urgency. In 2021, 72.8% of draft laws, 77.1% of draft government resolutions and 91.7% of ministerial decrees were submitted to the government without an RIA. These relatively high numbers seem to be unjustified, despite the COVID-19 pandemic, during which many regulations were drafted under the emergency procedure.
The Czech RIA system does not have a threshold test to determine whether RIA should be undertaken or whether impacts are estimated to be such that a simplified RIA would be sufficient. This affords the minister significant discretion in determining whether RIA is required for any given regulatory proposal.
The RIA’s level of detail is decided by the drafting authority. It should, however, respect the principle of proportionality. The RIA Board then assesses whether the level of detail is proportionate to the potential impacts of the proposal.
According to the General Guidance for Regulatory Impact Assessment, an RIA statement should contain a description of the problem, a justification for the need for a legislative solution, identification of the affected subjects and the results of the consultation. Impacts on the following should be assessed:
The guidance is perceived by some civil servants as rather complex and difficult to understand (Office of the Government of the Czech Republic, 2020[40]).
Methodological assistance in the process of developing individual RIAs might be provided by the RIA Unit, which is also responsible for co-ordinating the RIA process (see the section on regulatory oversight). The RIA Unit also provides comments on the quality of individual RIAs as part of the inter-ministerial comment procedure.
According to the General Guidance for Regulatory Impact Assessment, the final stage of the RIA process should be an ex post evaluation of regulatory impacts. The discrepancies between the real impacts and the intended ones should be evaluated at this stage. This evaluation is, however, only carried out very exceptionally due to the lack of enforcement (nobody is checking whether such an evaluation actually took place), lack of demand (nobody is asking for such an evaluation), relatively high resistance among civil servants and a lack of analytical capacities.
According to interviews as well as evaluations conducted by the government and independent think tanks, the quality of RIAs in the Czech Republic is relatively low. RIAs are often processed only formally to justify a decision that has already been taken. One of the biggest challenges is identifying a problem and setting objectives, which are too general. The impacts are often not identified properly and seldom quantified. The alternatives are often selected with minimum differences and non-regulatory solutions are rarely considered [see, for example, Münich (2019[54]) and Eršil (2018[53])].
Better using domestic institutions and tools to advance EIDM
The public administration plays a central role in advancing EIDM. In addition to the previously mentioned roles concerning capacities and skills, at a more practical level, the public administration is responsible for generating much of the data that forms the basis for EIDM. However, data cannot be generated if institutions are not in place and relevant tools are not utilised. An evaluation culture is not yet fully embedded in the Czech administration. In particular, although with some notable exceptions, evaluations are generally not conducted, and when they are, their impact is limited as they are not used to help reformulate policies.
A severe lack of evaluation is a critical evidence gap
EIDM is not linear; a decision taken at a single point in time lasts indefinitely. Circumstances change (both unintentionally and unexpectedly), and policies should be periodically evaluated to ensure that they remain in the public interest over time (OECD, 2020[15]). Some administrative reforms have been called for in the current political climate. However, implementation and follow up have generally tended to be weak within the Czech administration. For example, the vast majority of public strategies do not have an obligation to supply evidence to monitor their implementation. Notwithstanding the general lack of measurement and evaluation across the Czech Republic (see below), some evaluations have been conducted. However, they have tended to be limited to process evaluations, which largely assess whether certain outputs (e.g. reports, etc.) have been delivered on time. Even fewer public strategies have been subject to impact evaluations to assess whether they are delivering as originally intended (see Chapter 2). The OECD heard instances where there was little political appetite to provide any evidence, as a decision had already been taken. On the other hand, some decision-makers were more open to basing their decisions on available evidence (Box 3.10).
Between the 1990s and 2010s, gambling became a very serious social issue in the Czech Republic, with a steep rise of addictions and sociopathological issues connected to gambling due to, among other things, a large number of casinos and gambling machines, disproportionate to other European Union countries. Regulation from the 1990s did not address those problems at many levels, including municipalities’ competences in permitting, the arrival of new technologies, etc.
The situation was perceived as one of the most important social issues by the Czech public; nevertheless, attempts to introduce new regulation failed several times.
The Ministry of Finance started preparing a new Gambling Act in 2014 and decided to work with an expert team including external experts to base the new regulation on available evidence and data. This process led to the approval of the Gambling Act in 2016, which entered into force in 2017.
An EU-funded project supported the regulatory impact assessment (RIA) process to create model ex ante RIA that would serve as a best practice example for other ministries. Initially, the project team faced strong resistance among officials from the Ministry of Finance, but the preliminary results, including the assessment of the social costs of gambling addictions, helped to demonstrate that such analysis can contribute to better results.
One regulatory objective was to reduce the social risks of gambling and their impact. An ex ante analysis of the potential impacts of all the identified measures was carried out based on evidence using other countries’ experience. A combination of measures was chosen as the preferred solution based on the measurement of impacts, especially in terms of reducing social costs. The quantification included such costs as the financial costs of personal bankruptcy, loss of productivity costs (work, home), unemployment costs (job search), criminality and legal costs (police, court), personal and family costs (emotional costs), costs of medical treatment, etc.
A behavioural insights approach was also used during the process to identify some cognitive biases, such as optimism bias and the illusion of control leading to an overestimation of the probability of success. In addition, experiments were carried out testing individual solutions. It was found that warnings before the game do not affect players’ behaviour, but limits (administrative and self-limiting) and notifications of their status during the game do (e.g. reduced bets and increased time between spins).
Many stakeholders were consulted during the process, including gambling providers’ associations, non-governmental organisations, municipalities, the financial administration, the addictological community, banks, Internet service providers, the European Commission, etc.
The ex post evaluation carried out in 2021 showed that the 2016 gambling regulation is fully functional and fulfils its purpose. Among other things, it also protects players from the risk of gambling addiction due to a number of adopted addictological and self-limiting measures. In addition, the Register of Excluded Persons includes more than 200 000 people. In particular, as a result of the new gambling regulations, the number of slot machines has fallen by 62% since 2017, from 53 554 to 20 269. In the years 2013-14, there were approximately 115 000-130 000 people at high risk of problem gambling. In 2017-19, it was estimated at approximately 60 000-74 000.
Source: Based on a presentation by the Czech Office of the Government at the RegWatchEurope Workshop on Garnering Political Support, Prague, 8 June 2022.
One of the reasons participants suggested that public policy evaluation has not taken off is that undertaking evaluations is viewed as an administrative burden. Given the dearth of public policy evaluations, the wider Czech public administration stands to benefit from the culture and learnt experience of monitoring and evaluations undertaken as part of its membership to the European Union. Specific Czech entities could expand their scope, as has happened in a number of OECD countries, to more systematically embed a culture of review and evaluation across the civil service. The recently adopted OECD Recommendation of the Council on Policy Evaluation calls on member countries to foster a culture of learning and accountability through the promotion of evaluations (Box 3.11).
The OECD Recommendation of the Council on Public Policy Evaluation calls on member countries to institutionalise public policy evaluation from a whole-of-government perspective. Evaluations should be systematically carried out then used as part of continual improvement in public policy. Fostering a culture of learning and accountability is crucial to promote the necessary demand for, and ownership of, evaluations – both within and outside of the executive.
Evaluations must be actively planned, designed and managed to be timely and proportionate to the intended objectives. In particular, member countries should:
Plan evaluations early, by building provisions for evaluations into public interventions from the start, to improve their design, collect data on their implementation, and ensure that evaluation results are robust and available in a timely fashion.
Design and implement evaluations that are proportionate and appropriate for the likely use, by adapting the aim, scope and analysis of the evaluation, and its format and resources, to the needs of its primary users and the types of intended uses.
Engage relevant stakeholders in the evaluation process from the outset to create ownership for change and trust in evaluation results.
Build public sector skills for evaluation by conducting regular training; recruiting and retaining employees with the adequate skills; or collaborating with academia, the private sector and other jurisdictions to improve the availability of these skills.
Ensure the availability of high-quality, timely, accessible, disaggregated and reusable results, performance and administrative data for policy evaluation.
Provide institutions with appropriate resources to manage, carry out and use policy evaluations.
The necessary institutional arrangements require member countries to:
Provide guidance on when to conduct evaluations and the type of evaluation needed to adapt their timing to feed into decision-making processes, focus the analysis where it is most needed, co-ordinate efforts for cross-sectorial evaluations and avoid overlaps.
Incorporate the use of evaluation results into decision-making, including through the policymaking and budgetary processes.
Establish follow-up mechanisms for decision-makers to respond to the results of evaluations by defining a course of action where relevant and assigning responsibilities for implementing and tracking recommendations.
Make the result of evaluation findings and recommendations public by default.
Tailor the way evaluation evidence is presented and communicated to its potential users in terms of timing, communication channel, format and messaging by developing a dissemination strategy.
Make use of evidence synthesis methodologies to aggregate evaluation findings and systematically assess them.
Source: OECD (2022[55]).
The creation of a system of public administration measurement and evaluation largely emanated from requirements imposed by the European Commission to access specific funds (see below). As was noted in the Strategic Framework 2014-2020, the Czech Republic did not have a uniform evaluation system of public administration which at regular intervals would provide information about their actual condition, enable a comparison of the respective components, allow for correction in activities relating to the development of public administration and assess the expected development of the public administration (Ministry of the Interior of the Czech Republic, 2014[31]).
The Evaluation Unit within the Ministry for Regional Development plays a leading role in evaluations throughout the Czech administration. It has produced methodological guidance, which has been supplemented with practical aspects from its annual conference series, along with a repository of evaluations undertaken to date. The conference series is an open way to discuss aspects of evaluations that have been successful, and to work together to overcome the remaining challenges. It also has a demonstrative effect on the rest of the administration and fosters an environment of shared experiences.
Given the general absence of monitoring and evaluation elsewhere, work was undertaken to devise a methodology for assessing the functioning of the Czech public administration, determine measurement indicators – including through a pilot project – and publish the results under six broad headings via the Ministry of the Interior’s annual reports. The measurement and evaluation system was updated in 2020 and changed some of the measured indicators, albeit under the same six broad headings.
Two challenges were identified for creating the measurement and evaluation system of the Czech public administration. The first related to the unavailability of some data for the purposes of establishing the monitoring and evaluation regime. The second was reluctance by various Czech public bodies to share data across the administration (Ministry of the Interior of the Czech Republic, 2017[49]). These difficulties resulted in some changes to the indicators that were used to measure the performance of the Czech public administration. As a result, the measurement and evaluation system no longer corresponds as much to principles of general public management but rather strives to set appropriate trends (some with target values) that the Czech public administration should respect and strive to fulfil (Ministry of the Interior of the Czech Republic, 2020[56]).
One of the starkest results is that less than one-quarter of central state administration bodies consistently apply EIDM principles. The indicator is defined as those bodies that base their policies on proper analytical evaluation, have a data platform for data sharing within the organisation, regularly evaluate policies and strategies under their responsibility, have a functioning analytical team, and organise regular training for analysts (Ministry of Interior of the Czech Republic, 2022[17]).
Results to date indicate that digital service provision and use are increasing, although data are highly variable depending on the indicator. For instance, digital submissions via the data box increased by nearly 5.5 million between 2019 and 2020 (which amounts to approximately 3 transactions per citizen), whereas two more institutions began publishing open data in the same period, which still only represents a fraction of the total number of government institutions across both levels of government in the Czech Republic.
The indicators currently used to assess the performance of the Czech public administration could be improved. For instance, indicators relate to the number of laws created, amended or repealed. Several methodological issues focus on the volume of laws. First, in any given year, there may be significant external factors that affect the number of laws created, e.g. elections. In fact, the most recent annual report noted a large increase in the number of laws due to the COVID-19 pandemic (Ministry of Interior of the Czech Republic, 2022[17]). Second, the indicator is open to manipulation. Governments could simply consolidate many pieces of legislation into a single act, such that the number of acts passed would be less. Not only would this yield spurious improvement results, but it may also make it harder for citizens and businesses to locate legal obligations. From an EIDM perspective, a superior indicator would attempt to capture the quality of laws rather than the quantity. Any changes to the current set of indicators could potentially benefit from the methodological guidance for indicators, evaluations and publicity (Ministry for Regional Development of the Czech Republic, 2022[57]).
Under the European Union 2014-2020 programming period, EU member states were, for the first time, obliged to conduct evaluations to assess the effectiveness, efficiency and impact of the objectives of each programme as part of the Cohesion Policy. Cohesion Policy rules apply to the European Regional Development Fund, the European Social Fund and the Cohesion Fund (European Commission, 2015[58]). As the Cohesion Policy related to a broad range of activities, it follows that evaluations would be similarly broad-based.
To date, the Czech Republic has undertaken 262 evaluations under the European Social Fund as part of the 2014-2020 programme. More than half were implementation or progress related, with 27% being impact evaluations and 19% a mix of the two. Around 70% of the evaluations undertaken to date were undertaken by private consultants. Not only does this once again highlight a probable lack of analytical capacities to conduct evaluations, it also re-emphasises that such skills are not sought after or rewarded within the civil service. The outsourcing of evaluations also represents a missed opportunity to build and enhance learning capacities within ministries. More broadly, experiences could be widely shared across the administration to help strengthen and reinforce both the learning and technical aspects of conducting evaluations.
OECD research has highlighted the importance of supreme audit institutions in public accountability. However, these institutions have, for some time and in some countries, taken on a broader role more directly linked to decision-making by undertaking performance audits. Through this process, supreme audit institutions can provide insights to improve the functioning of processes and programmes, and foresight to aid governments in adapting to future trends and risks. Such evidenced-based contributions to addressing systemic issues can lead to better policy formulation, implementation and evaluation (OECD, n.d.[59]). The Supreme Audit Office of the Czech Republic undertakes this role as an important external check on government activity. Its most recent annual report notes a general lack of good design, monitoring and evaluation practices across a broad range of government programmes (Box 3.12). Reforming government programmes helps to ensure that scarce public resources are put to their highest value use. Improving service delivery can significantly strengthen and build resilience in public finances. It can also lead to better user experiences and trust in the regulatory framework.
The Supreme Audit Office noted the following selected issues based on the audits it undertook:
Insufficient statistical information for decision-making – The audit on reducing food waste and the distribution of food to people in need found that the Ministry of Agriculture only kept statistics on the number of projects supported and the amount provided without monitoring the benefits of the distributed aid. The ministry sourced information on the amount of food distributed from food banks. However, the information sources could not inform whether the money was used in the best possible way, as evidenced by the quantified costs of food aid distribution.
Programme design deficiencies
An absence of a risk-based approach – The audit of the Czech Social Security Administration’s expenditure on employees carrying out employer inspections was inefficient. The Czech Social Security Administration carried out checks every year on an across-the-board basis instead of focusing primarily on high-risk employers that were not fulfilling their legal obligations. In the audited period, about 76.5% of the performed inspections did not detect any shortcomings, while the total expenditure on these inspections amounted to CZK 1.2 billion.
Targets not based on actual need – An audit into the development and renewal of the material and technical assets of social services facilities found that the Ministry of Labour and Social Affairs set up programmes for establishing new residential beds in such a way that, in some cases, the targets set did not correspond to the actual need. At the end of 2016, for example, there were almost 92 000 unfulfilled requests for placement in new beds, but the ministry set a target to create just 605 additional new beds.
A lack of measurable output indicators
The audit of the National Strategy of Anti-drugs Policy for 2010-2019 noted that the audited ministries did not set measurable objectives or indicators in the drugs policy programmes allowing an assessment of their actual benefit, especially as regards prevention, risk reduction and reducing the availability of drugs for young people.
An audit of the Ministry of Industry and Trade’s CZK 75 billion investment incentives for nearly 20 years had no information showing how the investment incentives contributed to the Czech Republic’s economic development, even though this was supposed to be their primary purpose.
Source: Czech Republic Supreme Audit Office (2022[60]).
Over time, linkages could be made with government priorities to ensure that major programmes are subject to adequate monitoring and evaluation. Such evaluations can also form an ongoing series of reference documents for policymakers to learn what policy design and implementation processes worked well and where improvements can be made.
The Czech Republic currently does not conduct budgetary spending reviews. A pilot project carried out between the Ministry of Finance and the OECD identified barriers to more effective implementation of spending reviews in the Czech Republic, coupled with a technical implementation note. The pilot project is expected to be finalised soon and lessons from the exercise are anticipated to be integrated into a broader strategy on spending reviews (Ministry of Finance of the Czech Republic, 2022[61]). It is expected that a new unit within the Ministry of Finance will soon start undertaking spending reviews. Forthcoming OECD guidance can assist the Czech Republic in its efforts to embed spending reviews across the administration (OECD, forthcoming[62]). If spending reviews become well-established in the Ministry of Finance, they have the potential to identify issues and improve the effectiveness of government spending. Civil servants should be able to bring to bear the policy learnings from such reviews into policy (re)design and decision-making more broadly. Over time, capacity and experience of conducting spending reviews should grow.
Ex post reviews of existing regulations
As in most OECD countries, reviews of existing regulations in the Czech Republic focused for many years on reducing regulatory burdens. The primary focus was on reducing burdens for businesses. The Czech Republic was one of the first countries to use the Standard Cost Model to measure administrative burdens stemming from regulations across the administration. The project started in 2005 with the goal of reducing administrative burdens for businesses by 20% by the end of 2010. The latest initiative in this regard was adopted by the government in July 2019. The Plan for Systemic Administrative Burden Reduction on business for 2019-2022 contains a list of simplification measures which is updated by ministries on an ongoing basis (it currently includes 196 measures, 110 of which have been implemented).4 More recently, the Ministry of the Interior has launched a project on reducing administrative burdens on citizens and public sector organisations. In November 2022, the ministry also suggested abolishing more than 10 000 regulations that are considered to be outdated or obsolete.
Such projects, while useful, are, however, usually not considered as systemic ex post reviews of existing regulations, as they do not focus on the actual performance of regulations in terms of achieving their goals. A corollary is that reviews can be used to improve the allocation of scarce resources to worthwhile public activities. Reviews can identify areas of potential public finance savings, for example by adopting a more risk-based approach to compliance and enforcement based on the observed behaviour of market participants. Reviews can additionally identify situations where programmes are no longer in the public interest and make recommendations for improving them, saving financial resources of both governments and regulated entities.
Guidelines on conducting ex post assessments in the Czech Republic were published in 2018.5 However, they are voluntary, so officials are not obligated to follow them. In practice, evaluations are seldom undertaken (OECD, 2021[5]). This is despite the fact that the focus of ex post assessments is to review regulations that were subject to ex ante RIA. The proportion of regulatory proposals subject to RIA in recent years was around 23% (OECD, 2021[7]). An ex post assessment system based on an obligation to review the same regulations that were subject to ex ante RIA is sound, and is also consistent with some other OECD countries’ approaches (OECD, 2021[5]). It ensures that the proposals with the greatest anticipated societal impacts – where those impacts are estimated based on the available information and evidence – are the ones that are assessed ex post to determine if the impacts materialised as originally planned. With the relatively high number of exemptions granted from the obligation to carry out RIA, it might be worthwhile to consider introducing a measure to make ex post reviews obligatory in cases where such an exemption from ex ante evaluation has been granted.
As described above, the government has committed to introducing a framework for systemic ex post assessments of existing regulations. The OECD has recently provided detailed guidance to governments about the core points for establishing an ex post review framework (Box 3.13). The proposal for such a framework, including necessary amendments to the Legislative Rules of the Government and the General Guidance for Regulatory Impact Assessment, is still being discussed in the inter-ministerial comment procedure. There seems to be relatively high resistance among ministries to introducing obligatory ex post reviews. The proposed framework seems to be generally in line with OECD best practice and its adoption would contribute to improving the regulatory framework in the Czech Republic. However, its successful implementation would very much depend on strengthening analytical capacities in the administration, which are crucial for the systemic assessment of existing regulations and policies.
The overarching principles for ex post assessments are that:
regulatory policy frameworks should explicitly incorporate ex post reviews as an integral and permanent part of the regulatory cycle
a sound system for the ex post review of regulation would ensure comprehensive coverage of the regulatory stock over time while “quality controlling” key reviews and monitoring the operations of the system as a whole
reviews should include an evidence-based assessment of the actual outcomes from regulations against their rationales and objectives, note any lessons, and make recommendations to address any performance deficiencies.
Specific principles relate to:
broad approaches to reviews: programmed reviews; ad hoc reviews; and ongoing stock management
key questions to be answered in reviews: appropriateness, effectiveness, efficiency and alternatives
Source: OECD (2020[15]).
All countries’ RIAs focus on the marginal effects estimated to emanate from specific regulatory proposals. However, this means that the system-wide or cumulative impacts from multiple regulatory proposals are often overlooked. Addressing cumulative impacts will become increasingly important given the cross-cutting nature of legislation and the degree of interconnectedness, both domestically and internationally (OECD, 2019[63]). Cumulative impacts can arise both from within and outside the country, and this is particularly important for the Czech Republic as a country with national, regional and municipal levels of government that is also a Member State of the European Union. Well-designed ex post assessment systems provide opportunities to not only assess cumulative impacts, but also to identify potential overlaps and duplication across regulatory areas and levels of government.
Regulatory oversight
Regulatory oversight is a critical component of a well-functioning regulatory system (OECD, 2021[5]). It is perhaps unsurprising that there is currently no regulatory oversight of ex post assessments in the Czech Republic given its current ad hoc nature. As the evaluation system matures and capacities further develop, it will become increasingly important to ensure that evaluations are robustly conducted and subject to appropriate scrutiny as to their quality.
In the Czech Republic, an independent watchdog – the RIA Board – is responsible for overseeing the quality of RIAs produced by individual ministries and other agencies with competences for drafting legislation. The RIA Board is one of the nine Commissions of the Legislative Council of the Government. The Legislative Council of the Czech Government is an advisory body of the Government for the Government’s legislative work. Members of the Council, as well as of the RIA Board, are external experts performing their work for symbolic compensation. They are nominated by the Minister responsible for the work of the Council. In the case of the RIA Board, there are no set rules on how members of the Board should be selected. In practice, they are chosen largely based on their expertise or the body they are representing, but there are no clear criteria on which interests should be represented and balanced in the board membership. Also, there is no formal procedure for replacing inactive members of the Board unless they step down.
The Legislative Council is responsible for assessing the quality of legal norms before these are submitted to the government for discussion/approval. The primary focus is, however, mainly on constitutionality and compliance of the norm with other existing domestic legal documents, international treaties and the EU law. The Council also evaluates the linguistic quality and coherence, the use of plain and understandable language, etc. The Council issues a formal statement, which is then submitted to the Government session together with the draft of the legal norm. It should reflect the discussion at the meeting of the Legislative Council and reflect comments of all Commissions of the Council, including the RIA Board.
The RIA Board looks specifically at the quality of individual impact assessments and their compliance with the RIA Guidelines in case of legislative drafts prepared by the executive (ministries and other state administration authorities with the power to draft legislation). The individual drafts are, with exceptions, discussed at the meetings of the RIA Board with the presence of representatives of the responsible authority (e.g. ministry) submitting the legislative draft. The Statement of the RIA Board is then communicated to the responsible authority and presented to the Legislative Council. The RIA Board’s opinion is not binding for the Legislative Council or for the government itself. It is, therefore, not exceptional that a legislative draft is approved by the government despite a negative statement from the RIA Board.
There are four types of conclusions made by the RIA Board in its opinions. The RIA Board recommends to the Legislative Council to:
Recommend to the Government to adopt the proposal after implementing changes suggested by the RIA Board
Interrupt the discussion on the draft and to recommend amending the report following substantive comments by the RIA Board
Recommend to the Government not to adopt the proposal given the substantive comments made by the RIA Board.
In addition, based on a request by a drafting authority, the RIA Board might provide advice to the responsible ministry on how to develop individual RIAs. This advice is not binding for the assessment by the RIA Board. The RIA Board is not expected to comment on the substance of the legislative draft, only on the quality and completeness of the accompanying analysis of impacts. This practice is similar to other regulatory oversight bodies in OECD countries.
An important change to the RIA oversight process was made in 2016 by amending the Legislative Rules of the Government. First, the RIA Report is no longer an integral part of the substantive intent of the law, the justification report or the justification (in case of a bylaw) as it was the case before. The RIA report has become a separate document which is still part of the dossier that is submitted to the government. In addition, the RIA Board Statement is now only reflected in the final statement of the Legislative Council (or its chair). Before 2016, it was published as a separate annex to the Legislative Council Statement. The last (and maybe the most important) change is the fact that, in case of a recommendation made by the RIA Board to amend the RIA report, the submitting authority is not obliged to amend the report and resubmit it for another round of discussion at the RIA Board meeting. This has led to a situation where submitting authorities do not feel a need to amend the RIA report, even in cases of negative opinions of the RIA Board and therefore the Board’s opinions being taken less seriously. As shown by Eršil (2018[53]), the number of RIAs that were resubmitted to the RIA Board after a negative statement dropped significantly after 2016. (In 2016, it was 6.1 times less than in the previous year; only one RIA was resubmitted in 2017 out of 9 with a negative statement and 8 statements recommending changes.) The RIA Board, therefore, doesn’t have an opportunity to evaluate the quality of final RIAs, even if they are redrafted. Research also shows that the number of RIAs that are redrafted following the recommendation of the Board has dropped as well, as the authorities do not feel the pressure of the RIA Board potentially stopping the legislative process (Eršil, 2018[53]).
The RIA Unit at the Office of the Government consists of seven civil servants who analyse RIA reports submitted by lawmakers before forwarding them for further scrutiny by RIA Board. The RIA Unit provides methodological guidance and organises workshops and seminars for civil servants who prepare impact assessments. It also runs the website ria.vlada.cz with all the documents and information lawmakers need to successfully prepare a RIA. However, according to a recent survey (Office of the Government of the Czech Republic, 2020[40]), 30% of respondents (those responsible for drafting RIAs) do not have experience receiving feedback from the RIA Unit and 30% evaluate this feedback negatively. This might be due to the lack of capacities of the Unit as well as the lack of downstream engagement by the RIA Unit experts with the ministries already at the stage of drafting legislation where it could proactively offer assistance with the RIA process.
Improving the institutional arrangements for EIDM
Develop analytical capacities in the centre of government as well as in line ministries. The newly created VAU can be used as a basis for the central unit, assuming it will be properly staffed with a sufficient number of skilled analysts. In the medium term, analytical units should be created in all ministries through hiring and training following the plan set in the 2030 Strategic Framework for the Czech Republic.
Make use of external scientific institutes, think tanks and ministries’ research institutes. The VAU and analytical units in line ministries should map what kind of internal and external analytical bodies, including research institutes and think tanks, are available in their field of competence. Those capacities should be better exploited by strengthening co-operation with such bodies, e.g. through research grants.
Incorporate public access to data into the current government proposal on public data management, building on open data principles. Data should, wherever possible, be open and designed with sharing and interoperability (including with data outside of government) in mind. Implementation should include monitoring, such as usage statistics and metrics in the state of the public administration annual reports.
Increasing EIDM in the Czech Republic
Ingrain EIDM better in the public administration. The Office of the Government could take a co-ordinating role to ensure EIDM principles are put into practice throughout the public administration. Political commitment will be an important signal to ensure that civil servants view EIDM as an integral part of their work. Analytical capacities across the Czech administration need further investment for EIDM to flourish. A co-ordinated approach will be required from all relevant bodies.
Strengthen the role of RIA in the regulation-making process. Reinforce regulatory oversight of adherence to the current guidelines by strengthening the role of the RIA Board. Re-establish the statements of the RIA Board as a separate annex to the statement of the Legislative Council. An obligation to redraft a RIA after a negative statement of the Board should be reintroduced. An option of making the RIA Board independent from the Legislative Council could be considered. Publish ministerial decisions in determining whether regulatory proposals are required to be subject to RIA with reasons for exempting proposals from RIA. The RIA Unit in the Office of the Government should also function as a secretariat to the RIA Board, pre-screening legislative proposals submitted to the government at the stage of the inter-ministerial comment procedures and identifying the most problematic ones for the attention of the Board, helping to organise the work of the Board, etc. Regularly adapt the RIA processes and methodologies to reflect long-term societal challenges (e.g. sustainable development and climate change, innovation, just digital transformation, etc.).
Establish systematic monitoring and evaluation of government interventions as a core part of the public administration to understand what has worked, what has not and share practices across government. Building on work undertaken under the guises of the European Social Fund, general guidance should be provided to the public administration that can then be supplemented by entity-specific material if necessary. The production of the guidance could be jointly led by the Ministry of the Interior and the Office of the Government, which would be jointly responsible for implementing, monitoring and reviewing it. Additionally, political support and appropriate resourcing are needed to ensure that an evaluation culture permeates throughout the administration. Review the appropriateness of indicators selected for the state of the public administration by the Ministry of the Interior to ensure, wherever possible, they are based on outcomes and do not overlap.
Build a central interactive consultation portal where all members of the public can comment on draft regulatory proposals. Over time, the platform should be expanded to inform stakeholders of forthcoming consultations and be used for early-stage consultations. eKlep can be used as a basis for such a platform. Undertake training to improve the consultation culture and willingness to engage with affected stakeholders. Strengthen oversight over compliance with government guidelines for public consultations. Such a specific role could be given the newly established Council for Citizens and Stakeholders Participation and the RIA Board should promote the consultation culture as part of overseeing the quality of the RIA process.
Make guidance on conducting ex post assessments mandatory for all officials. Over time, the ex post assessment system should include an oversight function and expand beyond its current scope to ensure that cumulative impacts can be meaningfully assessed.
References
[2] Crowley, K. (ed.) (1986), The Collected Scientific Papers of Paul A. Samuelson, Massachusetts Institute of Technology, Cambridge, MA.
[60] Czech Republic Supreme Audit Office (2022), Annual Report 2021, Czech Republic Supreme Audit Office, Prague, https://www.nku.cz/assets/publications-documents/annual-report/annual-report-2021.pdf.
[37] Damschroder, L. et al. (2009), “Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science”, Implementation Science, Vol. 4/Article 50, https://doi.org/10.1186/1748-5908-4-50.
[50] Department of Sustainable Development (2021), Evaluation of Strategic Management for Sustainable Development, Ministry for Regional Development of the Czech Republic, Prague, https://www.cr2030.cz/wp-content/uploads/2021/02/Evaluace-Strategick%C3%A9ho-r%C3%A1mce-%C4%8Cesk%C3%A1-republika-2030_2017-2019_final.pdf.
[53] Eršil, L. (2018), Evaluation of the Impacts of Regulation in the Legislative Process in the Czech Republic, thesis, Faculty of Social and Economics, Jan Evangelista Purkyne University, Ústí Nad Labem, https://theses.cz/id/fgnau6.
[24] European Commission (2022), European Public Administration Country Knowledge, Country Brief 2021, Czechia, Directorate-General for Structural Reform Support, Public Administration and Governance, Publications Office of the European Union, Brussels, https://ec.europa.eu/info/system/files/2022-european-semester-country-report-czechia_en.pdf.
[58] European Commission (2015), “Evaluation of cohesion policy in the member states”, web page, https://cohesiondata.ec.europa.eu/stories/s/Evaluation-of-cohesion-policy-in-the-Member-States/suip-d9qs.
[38] Godkin, L. and S. Allcorn (2008), “Overcoming organizational inertia: A tripartite model for achieving strategic organizational change”, Journal of Applied Business and Economics, Vol. 8/1, pp. 83-95, http://www.na-businesspress.com/Godkin.pdf.
[6] Government of the Czech Republic (2022), Policy Statement of the Government, Government of the Czech Republic, Prague, https://www.vlada.cz/cz/programove-prohlaseni-vlady-193547.
[14] Government of the Czech Republic (2017), Strategic Framework Czech Republic 2030, Government of the Czech Republic, Prague, https://www.vlada.cz/assets/ppov/udrzitelny-rozvoj/projekt-OPZ/Strategic_Framework_CZ2030.pdf.
[13] Government of the Czech Republic (2016), General Principles for Impact Assessment Regulation (RIA), Government of the Czech Republic, Prague, https://ria.vlada.cz/wp-content/uploads/Obecn%C3%A9-z%C3%A1sady-pro-RIA-2016.pdf.
[52] Government of the Czech Republic (1998), Legislative Rules of Government (as amended), Government of the Czech Republic, Prague, https://www.vlada.cz/assets/jednani-vlady/legislativni-pravidla/LEGISLATIVNI-PRAVIDLA-VLADY_platne-od-1-2-2018.pdf.
[36] Hrstková, J. (2022), “Politicians fear it will be discovered that their policies are not working”, interview of Deputy Minister for Science, Research and Innovation Štěpán Jurajda, Hospodářské noviny, https://archiv.hn.cz/c1-67089730-politici-se-obavaji-ze-se-prijde-na-to-ze-jejich-politika-nefunguje.
[41] Langer, L., J. Tripney and D. Gough (2016), The Science of Using Science: Researching the Use of Research Evidence in Decision-making, EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London, London, https://eppi.ioe.ac.uk/cms/Portals/0/PDF%20reviews%20and%20summaries/Science%202016%20Langer%20report.pdf?ver=2016-04-18-142701-867.
[42] Michie, S., M. van Stralen and R. West (2011), “The behaviour change wheel: A new method for characterising and designing behaviour change interventions”, Implementation Science, Vol. 6/1, pp. 6-42, https://doi.org/10.1186/1748-5908-6-42.
[57] Ministry for Regional Development of the Czech Republic (2022), Methodological Guidelines for Indicators, Evaluations and Publicity, Ministry for Regional Development of the Czech Republic, Prague, https://www.dotaceeu.cz/cs/evropske-fondy-v-cr/kohezni-politika-po-roce-2020/metodicke-dokumenty/metodicke-dokumenty-v-gesci-mmr-cr/metodicky-pokyn-indikatory,-evaluace-a-publicita.
[47] Ministry for Regional Development of the Czech Republic (2018), Methodology for the Preparation of Public Strategies, Ministry for Regional Development of the Czech Republic, Prague, https://www.mmr.cz/getmedia/8e499918-ede3-45ed-864e-051764ac05b5/Metodika-pripravy-verejnych-strategii-plna-verze.pdf.aspx?ext=.pdf.
[51] Ministry for Regional Development of the Czech Republic (2017), Typology of Strategic and Implementing Documents: Methodological Recommendation, Ministry for Regional Development of the Czech Republic, Prague, https://www.mmr.cz/getmedia/a8e367ae-8c84-48f2-9ce4-5484e4d5de52/Typologie-strategickych-a-provadecich-dokumentu_final.pdf.
[45] Ministry for Regional Development of the Czech Republic (2015), “Evaluation conference”, https://dotaceeu.cz/cs/evropske-fondy-v-cr/narodni-organ-pro-koordinaci/evaluace/evaluacni-konference.
[61] Ministry of Finance of the Czech Republic (2022), Convergence Programme of the Czech Republic, Ministry of Finance of the Czech Republic, Prague, https://ec.europa.eu/info/sites/default/files/convergence_programme_czechiaapril_2022_en.pdf.
[17] Ministry of Interior of the Czech Republic (2022), Annual Report on the State of Public Administration in the Czech Republic in 2021, Ministry of Interior of the Czech Republic, Prague, http://kvalitavs.cz/wp-content/uploads/2021/07/Syst%c3%a9m-m%c4%9b%c5%99en%c3%ad-a-hodnocen%c3%ad-ve%c5%99ejn%c3%a9-spr%c3%a1vy-za-rok-2020.pdf.
[64] Ministry of Interior of the Czech Republic (2010), A Manual for Involving the Public in the Preparation of Government Documents, https://www.vlada.cz/assets/ppov/lrv/ria/metodiky/Manual-pro-zapojovani-verejnosti.pdf.
[30] Ministry of the Interior of the Czech Republic (2021), Fulfillment Information of the Implementation Plans for the Strategic Framework for the Development of Public Administration 2020, Ministry of the Interior of the Czech Republic, Prague, https://www.mvcr.cz/soubor/informace-o-plneni-implementacnich-planu-strategickeho-ramce-rozvoje-verejne-spravy-2020-k-15-cervnu-2021.aspx.
[56] Ministry of the Interior of the Czech Republic (2020), Third Evaluation Report on the Strategic Development of Public Administration in the Czech Republic 2014-2020, Ministry of the Interior of the Czech Republic, Prague, https://www.mvcr.cz/soubor/treti-evaluacni-zprava-k-plneni-strategickeho-ramce-rozvoje-verejne-spravy-ceske-republiky-pro-obdobi-2014-2020.aspx.
[48] Ministry of the Interior of the Czech Republic (2019), Client-oriented Public Administration 2030, Ministry of the Interior of the Czech Republic, Prague, https://www.mvcr.cz/soubor/client-oriented-public-administration-2030.aspx.
[8] Ministry of the Interior of the Czech Republic (2019), Risk Register to the Concept of Client-oriented Public Administration 2030, Ministry of the Interior of the Czech Republic, Prague, https://www.mvcr.cz/soubor/prilohy-ke-koncepci-klientsky-orientovane-verejne-spravy-2030-a-akcnimu-planu-na-leta-2021-2023.aspx.
[49] Ministry of the Interior of the Czech Republic (2017), Second Evaluation Report to Implement the Strategic Framework for Public Development Reports of the Czech Republic for the Period 2014-2020, Ministry of the Interior of the Czech Republic, Prague, https://www.mvcr.cz/soubor/druha-evaluacni-zprava-k-plneni-strategickeho-ramce-rozvoje-verejne-spravy-ceske-republiky-pro-obdobi-2014-2020.aspx.
[31] Ministry of the Interior of the Czech Republic (2014), Strategic Framework for the Development of Public Administration in the Czech Republic for the Period 2014-2020, Ministry of the Interior of the Czech Republic, Prague, https://www.mvcr.cz/soubor/strategic-framework-for-the-development-of-public-administration-in-the-czech-republic-for-the-period-2014-2020.aspx.
[54] Münich, D. (2019), “Ill-conceived laws cause enormous damage. We are breaking the rules of good governance”, Deník N, https://denikn.cz/218892/nedomyslene-zakony-zpusobuji-obrovske-skody-svejkujeme-pravidla-dobreho-vladnuti/?ref=list.
[9] Newman, K., C. Fisher and L. Shaxson (2012), “Stimulating demand for research evidence: What role for capacity-building?”, IDS Bulletin, Vol. 43/5, pp. 17-24, https://doi.org/10.1111/j.1759-5436.2012.00358.x.
[55] OECD (2022), Recommendation of the Council on Public Policy Evaluation, OECD, Paris, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0478.
[26] OECD (2021), “Data as a strategic asset for the public sector”, in Government at a Glance 2021, OECD Publishing, Paris, https://doi.org/10.1787/785cb67f-en.
[46] OECD (2021), “Digital by design: Strengthening co-ordination and skills to foster digital transformation”, in Government at a Glance 2021, OECD Publishing, Paris, https://doi.org/10.1787/85ec7a68-en.
[27] OECD (2021), “Digital government: Progress towards digital competence and maturity”, in Government at a Glance 2021, OECD Publishing, Paris, https://doi.org/10.1787/2bed4623-en.
[7] OECD (2021), Indicators of Regulatory Policy and Governance Survey, unpublished.
[19] OECD (2021), Mobilising Evidence at the Centre of Government in Lithuania: Strengthening Decision Making and Policy Evaluation for Long-term Development, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/323e3500-en.
[5] OECD (2021), OECD Regulatory Policy Outlook 2021, OECD Publishing, Paris, https://doi.org/10.1787/38b0fdb1-en.
[23] OECD (2020), OECD Open, Useful and Re-usable data (OURdata) Index: 2019, OECD Publishing, Paris, https://www.oecd.org/gov/digital-government/policy-paper-ourdata-index-2019.htm.
[1] OECD (2020), Building Capacity for Evidence-Informed Policy-Making: Lessons from Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/86331250-en.
[21] OECD (2020), Improving Governance with Policy Evaluation: Lessons From Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/89b1577d-en.
[33] OECD (2020), Mobilising Evidence for Good Governance: Taking Stock of Principles and Standards for Policy Design, Implementation and Evaluation, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/3f6f736b-en.
[18] OECD (2020), One-Stop Shops for Citizens and Business, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/b0b0924e-en.
[4] OECD (2020), Regulatory Impact Assessment, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/7a9638cb-en.
[15] OECD (2020), Reviewing the Stock of Regulation, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/1a8f33bc-en.
[63] OECD (2019), 12th OECD Conference on Measuring Regulatory Performance: Agile Regulatory Oversight for the 21st Century, OECD, Paris, https://www.oecd.org/gov/regulatory-policy/Conference-proceedings-MRP-Canada.pdf.
[35] OECD (2019), The Path to Becoming a Data-Driven Public Sector, OECD Digital Government Studies, OECD Publishing, Paris, https://doi.org/10.1787/059814a7-en.
[3] OECD (2018), OECD Regulatory Policy Outlook 2018, OECD Publishing, Paris, https://doi.org/10.1787/9789264303072-en.
[29] OECD (2018), Open Government Data Report: Enhancing Policy Maturity for Sustainable Impact, OECD Digital Government Studies, OECD Publishing, Paris, https://doi.org/10.1787/9789264305847-en.
[44] OECD (2017), “Core skills for public sector innovation”, in Skills for a High Performing Civil Service, OECD Publishing, Paris, https://doi.org/10.1787/9789264280724-6-en.
[11] OECD (2017), Recommendation of the Council on Open Government, OECD, Paris, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0438.
[16] OECD (2012), Measuring Regulatory Performance: A Practitioner’s Guide to Perception Surveys, OECD Publishing, Paris, https://doi.org/10.1787/9789264167179-en.
[12] OECD (2012), Recommendation of the Council on Regulatory Policy and Governance, OECD Publishing, Paris, https://doi.org/10.1787/9789264209022-en.
[62] OECD (forthcoming), OECD Best Practice Principles on Spending Reviews, OECD Publishing, Paris, forthcoming.
[59] OECD (n.d.), OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/22190414.
[10] OECD (n.d.), OECD Regulatory Policy Working Papers, OECD Publishing, Paris, https://doi.org/10.1787/24140996.
[40] Office of the Government of the Czech Republic (2020), Report From a Survey on the Practice of Regulatory Impact Assessment 2019, Office of the Government of the Czech Republic, Prague, https://ria.vlada.cz/wp-content/uploads/Zpr%C3%A1va-k-%C5%A1et%C5%99en%C3%AD-praxe-RIA-2019.pdf.
[22] Results for All (2017), 100+ Government Mechanisms to Advance the Use of Data and Evidence in Policymaking: A Landscape Review, Results for America, Washington, DC, http://results4america.org/wp-content/uploads/2017/08/Landscape_int_FINAL.pdf.
[39] Shaxson, L. (2019), “Uncovering the practices of evidence-informed policy-making”, Public Money & Management, Vol. 39/1, pp. 46-55, https://doi.org/10.1080/09540962.2019.1537705.
[32] Statistics Denmark (2014), Data for Research, https://www.dst.dk/en/TilSalg/Forskningsservice#.
[34] Statistics Denmark (n.d.), “About Denmark’s Data Portal”, web page, https://www.dst.dk/en/TilSalg/Forskningsservice/danmarks-datavindue/om-danmarks-datavindue.
[28] Ubaldi, B. (2013), “Open Government Data: Towards Empirical Analysis of Open Government Data Initiatives”, OECD Working Papers on Public Governance, No. 22, OECD Publishing, Paris, https://doi.org/10.1787/5k46bj4f03s7-en.
[25] Ubaldi, B. and T. Okubo (2020), “OECD Digital Government Index (DGI): Methodology and 2019 results”, OECD Working Papers on Public Governance, No. 41, OECD Publishing, Paris, https://doi.org/10.1787/b00142a4-en.
[43] UK Government (2022), “What Works Network”, https://www.gov.uk/guidance/what-works-network.
[20] van Ooijen, C., B. Ubaldi and B. Welby (2019), “A data-driven public sector: Enabling the strategic use of data for productive, inclusive and trustworthy governance”, OECD Working Papers on Public Governance, No. 33, OECD Publishing, Paris, https://doi.org/10.1787/09ab162c-en.
Notes
← 1. See Government of the Czech Republic (2016[13]) and Ministry of the Interior of the Czech Republic (2010[64]).
← 2. According to a definition used by the Slovak government, such units should provide “analytical services to the management of its institution, so it would be able to make decisions that would be based on data and their analysis in case of sectoral and cross-cutting policies” (https://www.mfsr.sk/files/archiv/78/Metodika-budovania-analytickych-kapacit-v-statnej-sprave.pdf).
← 3. See the section on regulatory management tools.
← 4. https://www.mpo.cz/assets/cz/podnikani/regulace-podnikani-a-snizovani-administrativni-zateze/snizovani-administrativni-zateze-podnikatelu/2022/11/Zprava-o-vyvoji-PP-2021.pdf.
← 5. https://ria.vlada.cz/wp-content/uploads/Metodika-prezkumu-ucinnosti-pravnich-predpisu.pdf.