3. Integrating the multi-level governance dimension into the audit of decentralised policies in Brazil
This chapter describes how the federal, state and municipal courts of accounts in Brazil can develop a multi-level governance (MLG) assessment framework for the audit of decentralised policies. It begins by proposing a four-step model that details the concrete actions that audit institutions should follow in developing a framework for specific policy sectors. It goes on to present a generic MLG assessment framework developed by the OECD for Brazil that can be used as a guide for developing sector-specific frameworks. The chapter then describes maturity indicators on key dimensions of the MLG system that should be assessed during the audit. Finally, it examines how this four-step model could be applied to the education sector in Brazil, taking as key input the generic model.
The way in which multi-level governance (MLG) is structured – and its level of efficiency – are going to affect the design, implementation, and outputs of public policies with subnational impact. Effective multi-level governance can facilitate well-managed and co-ordinated public policies in decentralised policy areas and generate substantial savings. It is therefore important to have a clear overview of the capacities and resources at each level, to allocate responsibilities in the most effective way, and to develop co-ordination mechanisms between and across each level. Multi-level governance is all the more important in the context of federal countries, where sovereignty is shared between the federal government and the states and municipalities (see Chapter 1).
Brazil has a complex multi-level institutional and financial architecture. This complexity has implications for the overall performance of the decentralised system, as it poses a number of challenges for effective multi-level governance. To reach desired policy outcomes in Brazil, it is necessary to account for the linkages and the relationships between federal regulations and frameworks and municipal decision-making, but also between state and non-state actors involved in the design and implementation of social policies.
When conducting audits it is important to have knowledge about the system in which policies are operating. In the case of decentralised policies, analysis of the multi-level governance dimension is particularly crucial. An assessment of the multi-level governance allows a better understanding of the existing mechanisms and tools that are in place and potentially have an impact on the efficiency and effectiveness of policy delivery.
In order to better assess the effectiveness and efficiency of decentralised policies, auditors could integrate the MLG dimension into the audit process, once the audit topic has been selected and the relevance of MLG for this topic has been established (see Chapters 2 and 4).
For this purpose the OECD proposes to define maturity indicators that allow and facilitate identification of:
the key elements that should be analysed in-depth during the audit, including a brief explanation of why the analysis is important
the unit of analysis
the level of maturity for each of those elements
the sources of information that could be used to assess the level of maturity.
The OECD has developed a specific grid template for the maturity indicators that might be used. As will be mentioned below, the indicators to guide the audit should be defined during the pre-audit exercise.
Key steps in developing a framework for specific policy areas
The development of an assessment framework unfolds in several steps (see Figure 3.1). To maximise its usefulness, the collaborating audit entities are encouraged to design a framework for each policy area that will be audited. To facilitate this work, the OECD has developed a generic assessment framework, based on a general description of the Brazilian MLG structure. This generic framework identifies maturity indicators that could be integrated into the auditing of specific policy sectors.
1. Describe the multi-level governance system for a policy area and identify key dimensions that should be assessed during the audit
To develop an assessment framework for the audit of multi-level governance in a pre-selected policy area, it is important to describe in general terms how the MLG system works in that specific area. This description may allow auditors to identify the main MLG challenges faced by all levels of government.
The purpose of this step is not to perform a thorough and deep analysis of MLG, but rather to gain a comprehensive understanding of how the policy area to be audited works in a decentralised context.
For resources, audits institutions may:
Use as a reference “Making Decentralisation Work: A Handbook for Policy-makers”, which details 10 guidelines for effective decentralisation (OECD, 2019[1]), and the OECD Recommendation on Effective Public Investment across Levels of Government, which presents 12 principles for improving the MLG of public investments (OECD, 2019[2]). Both documents include a grid of self-assessment indicators to help policy-makers identify the key issues they should focus on. The documents also include good practices from OECD countries.
Establish a working group including representatives from federal, state and municipal courts of accounts that have experience and knowledge in the policy area that will be audited. This working group may discuss in detail how the policy works and the main challenges with regard to multi-level governance.
Organise discussions with experts on the subjects to be audited. These discussions should focus on i) understanding how the policy works, and ii) identifying the main audit questions
Review previous audits that have been performed in the policy area pre-selected, as well as any other relevant literature, to inform the description and identification of key issues.
When targeting the audit to a specific state or municipality, it is important to deepen the background focus on individual state or municipal governments. By examining the situation of given states or municipalities through national and international comparisons, the auditors will be able to identify the main challenges regarding multi-level governance that apply.
Having worked out a general description of the MLG system, audit institutions should identify the elements of the multi-level governance system that seem to be most crucial for efficient and effective policy delivery. The objective is to select the key issues and challenges of the MLG system that merit more in-depth assessment during the audit; the resources listed above should help in reaching consensus. Once the assessment framework is applied during the audit itself, different elements may arise that were not included in this first step. In that case, the assessment framework will need to be revised accordingly.
The OECD has developed ten guidelines for implementing decentralisation, which apply to both federal and unitary countries. The guidelines are more than just recommendations: they cover key issues of decentralisation, with the objective of helping policy-makers implement reforms. The report “Making Decentralisation Work: A Handbook for Policy-Makers” further details the rationale for each guideline, offers practical guidance, identifies pitfalls to avoid, highlights good practices, and includes a checklist for action. The ten guidelines are summarised below.
1. Clarify the responsibilities assigned to different government levels:
The way responsibilities are shared should be explicit, mutually understood and clear for all actors. Clarity does not mean that shared responsibilities should be avoided, as this is by definition impossible. Equally important is clarity in the different functions that are assigned within policy areas – financing, regulating, implementing or monitoring.
Clear assignment is critical for accountability, monitoring and effectiveness of investment and service delivery policies. The more a responsibility area is shared across different government levels, the greater clarity is needed to reduce duplication and overlaps.
Since multi-level governance systems are constantly evolving, a periodic review of jurisdictional assignments should be made to ensure flexibility in the system.
2. Ensure that all responsibilities are sufficiently funded:
Access to finance should be consistent with functional responsibilities. Division of financing responsibilities should ensure that there are no unfunded or underfunded assignments or mandates.
The formulas for determining central government transfers, grants and earmarked funds from the centre to lower levels of government should be transparent and nondiscretionary.
3. Strengthen subnational fiscal autonomy to enhance accountability:
Subnational governments should have a certain degree of autonomy in the design and delivery of their public service responsibilities within the limits set by normative regulations, such as minimum service standards.
Subnational governments need own-source revenues beyond grants and shared tax revenues – and they need to develop other sources to have a balanced basket of revenues.
4. Support subnational capacity building:
Central government should assess capacity challenges in the different regions on a regular basis. Policies to strengthen capacities should be adapted to the various needs of territories. Governments should seek to reinforce the capacities of public officials and institutions in a systemic approach, rather than adopting a narrow focus on technical assistance.
Staff training in the basics of local public financial management should be established. Open, competitive hiring and merit-based promotion should be ensured.
Special public agencies accessible to multiple jurisdictions should be encouraged in areas of needed expertise (e.g. regional development agencies, public-private partnership units).
5. Build adequate co-ordination mechanisms across levels of government:
Since most responsibilities are shared, it is crucial to establish governance mechanisms to manage joint responsibilities. Creating a culture of co-operation and regular communication is essential for effective multilevel governance and successful long-term reform. Tools for vertical co-ordination include for example dialogue platforms, fiscal councils, commissions and intergovernmental consultation boards, and contractual arrangements.
It is important to avoid multiplying co-ordination mechanisms with no clear role in the decision-making process.
6. Support cross-jurisdictional co-operation:
Horizontal co-ordination can be carried out using specific matching grants, and by promoting inter-municipal and interregional co-operation. Metropolitan governance should be promoted as well. The legal system at the national level should allow such tools.
Rural-urban partnerships should be promoted as a form of cross-jurisdictional collaboration to enhance inclusive growth by bringing multiple benefits, such as expanding those of agglomeration economies, to overcome co-ordination failures and strengthen capacity.
7. Strengthen innovative and experimental governance, and promote citizens’ engagement:
Citizens should be empowered through access to information. Ensure that elected local councils have the ownership and control of citizen participation and engagement initiatives.
Participatory budgeting has the potential to strengthen inclusive governance.
8. Allow and make the most of asymmetric decentralisation arrangements:
Asymmetric decentralisation should be supported by effective vertical and horizontal co-ordination mechanisms, and needs to go hand in hand with an effective equalisation system.
An asymmetric decentralisation approach should be based on dialogue, transparency and agreements among all main stakeholders, and be part of a broader strategy of territorial development.
The way asymmetric responsibilities are allocated should be explicit, mutually understood and clear to all actors. To the greatest extent possible, participation in an asymmetric arrangement should remain voluntary.
9. Consistently improve transparency, enhance data collection and strengthen performance monitoring:
National governments should develop performance-monitoring systems to track decentralisation and regional development policies: these need to remain simple with a reasonable number of requirements/indicators.
Higher-level governments need to monitor subnational performance in critical service areas based upon a minimum set of standardised indicators and provide timely feedback, as well as benchmark inter-local performance in service delivery.
Subnational governments need to be subject to higher-level regulations and fiscal rules to ensure fiscal discipline and fiscal sustainability.
10. Strengthen national regional development policies and equalisation systems, and reduce territorial disparities:
The equalisation programme must not be viewed as separate from the broader fiscal system, especially conditional transfers. Equalisation arrangements need to be carefully designed to promote the tax and development efforts of subnational governments. Fiscal equalisation policies need in particular to be accompanied by proactive regional development policies to offset the potential negative incentives of such systems.
Source: (OECD, 2019[3]).
In summary, the main result of this step should be a short report or paper prepared by audit institutions during the pre-audit exercise. The report should contain the inputs described above, along with the description of the main MLG challenges faced by all levels of government. It should include as a basis (but not be limited to):
i. An overview of the institutional structure in the policy area or sector.
ii. How the responsibilities are assigned and financed across levels of government, and the relative importance of subnational governments as actors in policy delivery. This also implies looking at the degree of legislative control of each level of government and the degree of decision-making power.
iii. The main co-ordination instruments and arrangements that are in place for the specific policy sector.
iv. Clear identification of key elements of the multi-level governance system that will be assessed more in detail during the audit (e.g. assignment of responsibilities, vertical and horizontal co-ordination, capacity-building processes, fiscal framework, stakeholder engagement, evaluation and monitoring, etc.).
In 2014, the OECD Regional Development Policy Committee endorsed the Recommendation on Effective Public Investment across Levels of Government. The Recommendation aims to help countries assess the strengths and weaknesses of their public investment governance capacity for regional development across all levels of government. It serves as a guide for setting priorities to improve the co-ordination mechanisms and capacities of subnational governments in the management of public investment.
The Recommendation sets out 12 principles grouped into three pillars of policy recommendations that represent three systematic challenges to efficiently managing public investment at both national and subnational levels. These principles cannot be seen in isolation; they offer a whole-of-government approach that addresses the roles of different levels of government in the design and implementation of critical and shared responsibility. All the principles are complementary and there is no hierarchy among them.
2. Define key maturity indicators for each of these dimensions
As a second step, the auditors should ascertain the development level of each of the key dimensions identified in the previous step, using the different qualitative maturity indicators. These correspond to specific and measurable aspects that auditors should consider when assessing performance of multi-level governance associated with the specific policy under evaluation. It is crucial that the indicators are clear and understandable to all audit entities involved; that they are consistent and do not overlap; and that they capture any complementary features of the MLG system.
For each maturity indicator, the assessment framework should specify:
The unit of analysis – Identify whether or not the indicator is intended to capture an issue in which all levels of governments (federal, states and municipalities) are involved. If the unit of analysis is the municipality, the indicator has to consider elements that can be controlled solely by the municipality.
The description and rationale – This category should specify the rationale for including the indicator in the framework, including a description of the criteria for describing the maturity levels, its main characteristics, and its relation to multi-level governance issues, using clear, concrete and plain language.
The source of information – The framework should also specify all the different and complementary sources of information that should be used during the assessment. This list is not exhaustive and may be changed and adapted during the audit process itself. The sources of information can be but are not limited to laws and regulations, fieldwork (for example, in the form of interviews or focus groups with different stakeholders), previous audits, policy and programme evaluations conducted by the policy owner or experts/academics, qualitative surveys and financial accounts.
Diagnostic question – Specify the main question that allows evaluating the degree of maturity for each indicator (see next step). This helps in clearly stating the main issue that the indicator intends to capture.
3. Specify the different levels of maturity for each indicator
As a third step, auditors have to set maturity levels corresponding to the different levels of accomplishment of each indicator. The OECD suggests that each indicator identifies three levels of maturity as follows:
1. Initial/Basic level – The first maturity level considers the basic actions that the unit(s) of analysis can take in the different indicators. It indicates that the conditions are not necessarily in place or not functioning well.
2. Satisfactory/Partial achievement – The second level of maturity reflects that the dimensions involved are better understood, bringing some level of organisation, standardisation and systematisation to practices. It signals that the unit of analysis has implemented some steps to advance but that improvements are still needed.
3. Optimised/good performance – The third level of maturity represents best practice for each indicator and reflects that the system in place works in a satisfactory way. This best practice indicates that the unit(s) of analysis has/have proactively implemented measures allowing them to perform at their best. This level might reflect good practices from different OECD countries.
To develop the descriptions for each maturity level, auditors ideally would take into account the three principles below:
Existence – To what extent does the practice/situation exist in the unit(s) of analysis (e.g. a state or a municipality)?
Quality – To what extent is the practice implemented as planned? To what extent does it yield expected results (i.e. contributing to the corresponding capacity and goal)? And, where applicable, to what extent is it tailored to different situations or audience?
Frequency – Where applicable, to what extent does the practice occur – e.g. continuously, regularly, or on an ad hoc basis?
The quality principle should prevail. Thus for individual criteria, even if the practices are regularly undertaken but there are deficiencies in implementation, or the results do not yield expected outcomes, the maturity level should be considered “partial” (level 2) or “basic” (level 1).
Each maturity level should include a clear and as detailed as possible description of each of the three levels to provide guidance and ensure consistency throughout the ranking process. Auditors can then assess the degree to which the situation of the unit of analysis corresponds most, in accordance with the various maturity levels.
4. Apply the MLG assessment framework to audit
Once the assessment framework has been defined, it can be used as the main reference to design and plan the audit process. The main result of this design phase is the audit plan, which usually includes audit objectives, scope, criteria, evidence collection and analysis techniques.
The design phase requires auditors first to consider assessing auditability and understanding what will be audited (INTOSAI, 2019[5]). In assessing auditability, auditors need to check whether it is possible to set audit criteria and whether the information required is likely to be available and can be obtained without difficulty. Understanding the topic or object to be audited is important in any type of audit, but in performance audits it can be crucial. They involve a continuous and cumulative process of gathering and assessing information at all stages of the audit, but in particular in the design phase, formulating and testing initial hypotheses and revisiting the auditability question. Good practice is to carry out the process in a pre-audit study.
In that respect, development of the MLG assessment framework for a specific decentralised policy area can indeed be considered a pre-audit or preliminary study. The SAI can also consider publishing this study not as an audit, but as an alternative product to highlight certain issues that may require the attention of the government and other stakeholders (see Box 3.3 for examples).
The National Audit Office of the United Kingdom
The National Audit Office of the United Kingdom (NAO) conducts investigations to establish facts that may point to potential concerns about public spending issues, such as service failures or financial irregularities. These concerns may be raised by MPs, the media or the public, or be identified through the work of the NAO itself.
These investigations uncover the facts rather than evaluating consequences as would audits, so they can report rapidly to address live and emerging issues and provide parliament with timely reports.
The investigation may give rise to lessons, which in turn contribute to NAO good practice guides and frameworks. For instance, the investigations of “Programmes not achieving their aims” – e.g. the Investigation into the Department of Energy & Climate Change’s loans to the Green Deal Finance Company and Investigation into Just Solutions International – contributed to the NAO Framework to Review Programmes.
The Netherlands Court of Audit
The NCA has a long-standing practice of publishing background reports. These reports are descriptions of usually complicated systems having to do with oversight and public accountability. They are meant to inform decision making in parliament.
For instance, in 2009 the NCA published an exploratory study of the system in place to supervise the stability of the financial system in the Netherlands. De Nederlandsche Bank (DNB) exercises this supervision. The report highlights tensions that are inherent in the supervisory system and draws attention to a number of critical themes.
In its strategic work on accountability and oversight of non-governmental organisations that execute a statutory task (e.g. schools, hospitals, environmental management, issuing car registration etc.), the NCA published a number of comparative studies and background reports between 1998 and 2009. The purpose of the reports was to clarify certain issues and develop criteria for good governance in these types of non-governmental bodies and arm’s-length institutions.
These reports not only inform parliamentarians, but also are used by academia and the leadership of these organisations.
Sources: (NAO, 2018[6]) (NAO, 2016[7]) (Algemene Rekenkamer, 2000[8]) (Algemene Rekenkamer, 2009[9]).
The rigour with which the MLG assessment framework has been developed will facilitate the next steps of designing the audit. The framework describes audit questions, audit criteria – in terms of maturity levels – and information to be collected in order to establish whether these criteria are met.
In case the audit is going to be a collaboration among different audit bodies, the audit design matrix, pertinent audit questions, criteria and information can be based on the MLG assessment framework, facilitating the division of work and at the same time ensuring internal consistency.
Following the methodology described in the previous section for Steps 1-3, the OECD has developed a generic assessment framework for multi-level governance in Brazil. This generic assessment framework may facilitate and guide audit entities in identifying which key MLG dimensions deserve deeper analysis (Step 1), where and how this information may be gathered (Step 2), and what features represent good practices based on OECD standards (Step 3).
The generic framework identifies maturity indicators that could be integrated into the auditing of specific policy sectors. It serves the purpose of guiding and facilitating the work of the audit institutions while preparing the assessment framework for specific policy sectors, including the selection of indicators. The assessment framework for education policy in Brazil, presented in the next section, was also developed taking this generic framework as a reference. Below are details of each step followed by the OECD to prepare it.
Description of the multi-level governance system and identification of key elements
The OECD first conducted a literature review to develop an overview and general description of the main decentralisation and multi-level governance features of Brazil. This review was based, among other things, on OECD reviews and databases, as well as academic papers and Brazilian law and regulations on decentralisation and multi-level governance in the country. The details of this overview were presented to the project partners in a background paper. A summary is given in Box 3.4.
Decentralisation in Brazil
The main features of the decentralisation system in Brazil, focusing on the local level, can be summarised as follows:
Brazil stands out as an exception among federal countries, with a more prominent role played by municipalities, a high degree of local autonomy and no hierarchical relationship between state governments and municipalities.
While Brazil is considered a highly decentralised country according to traditional indicators, the autonomy of state and local governments is somewhat limited by a rigid fiscal system and frequent overlap of responsibilities.
Both states and municipalities are important social and economic actors in Brazil. Subnational governments in Brazil accounted for 44% of total public expenditure and 19% of GDP, above the OECD average (40% and 16% respectively in 2016); specifically, municipalities accounted for 19.3% of total public spending in 2016.
The main expenditure areas of municipalities were education (26.6% of municipal expenditure), health (24.3%) and general public services (20.3%). Brazilian municipalities’ role in public investment has, however, been decreasing in the past year, from around 41.3% of total public investment in 2016 to 36% in 2018, and the overall public investment level remains low in Brazil compared with other Latin American and Caribbean countries (OECD, 2019[10]).
Tax and own-source revenue represent a significant share of subnational total revenue (78% for states and 34% for municipalities in 2016). There is also a system of equalisation transfers through the apportionment of revenue from tax collection by the Union among the federated entities. This mechanism aims to alleviate regional inequalities at the state and municipality levels. The State Participation Fund (FPE) and the Municipality Participation Fund (FPM) are the main fiscal equalisation transfers.
Most expenditure and revenue flows in Brazil are earmarked and a large share of expenditure is mandatory (Constitution of the Federative Republic of Brazil, Art. 212).
Multi-level governance in Brazil
Brazil has a highly complex, multi-level institutional and financial structure. This complex architecture has implications for the overall performance of the decentralised system, drawing a number of challenges for multi-level governance to work effectively.
At the vertical level, municipalities and state governments in particular have a relatively strong political and financial autonomy, considering the limitations stated above. At the horizontal level, Brazilian institutions tend to work mostly in-silo, meaning that each sectoral ministry has its own territorial vision and does not necessarily co-ordinate with other ministries. Besides, a large number of municipalities rely on non-state actors from the private sector and civil society to provide social services.
Several bodies dedicated to intergovernmental dialogue have been suspended in the recent years (e.g. the Ministry of Cities, and the National Council for Cities, which was in charge of implementing and monitoring the National Policy for Urban Development).
Two forms of multi-level arrangements are particularly common in Brazil: Contract agreements and federative pacts. In addition, some ministries have developed mechanisms on their own to ensure vertical co-ordination. This is the case for instance in the health sector, which is decentralised largely to state and municipalities.
Source: (Zapata and Lafitte, 2020[11]).
When analysing this overview in relation to the OECD guidelines for making decentralisation work (OECD, 2019[3]), it was clear that a generic framework for Brazil needed to include at least a focus on: i) the assignment of responsibilities and instances where these are unclear or overlapped; ii) the funding of those responsibilities to detect unfunded or underfunded mandates; iii) the co-ordination mechanisms in place to align priorities and investments; and iv) territorial disparities and the mechanisms in place to reduce them.
The OECD also undertook several discussions and exchanges with a diverse set of experts on multi-level governance and decentralisation in the country, including discussions with experts in the framework of an OECD mission to Brasilia. The OECD also benefited from the feedback and exchanges with the Federal Court of Accounts (Tribunal de Contas da União, TCU) and the state and municipal courts of accounts (Tribunais de Contas do Estado [TCEs] and Tribunais de Contas dos Municípios [TCMs], respectively) during missions or virtual conferences. In July 2019 for example, the TCU and courts of accounts (TCs) participated in a session to discuss the main multi-level governance challenges in Brazil. On this occasion, in addition to the elements previously highlighted, participants also noted that challenges linked to capacities of local governments, as well as those regarding access to data, transparency and accountability, were crucial issues in Brazil.
This exercise led to the OECD selecting six dimensions that raise key challenges for the efficient functioning of the multi-level governance system in Brazil (see below). They are:
1. assignment of responsibilities
2. funding of subnational responsibilities
3. capacities of subnational governments and capacity building
4. co-ordination among levels of government
5. performance monitoring and transparency
6. fiscal equalisation systems and regional policies to reduce territorial disparities.
These dimensions also combine different key factors identified in the 10 OECD guidelines on making decentralisation work, and the OECD Council Recommendations for Public Investment across Levels of Government. At the same time, as explained above, these six categories bring together the main points of interest for Brazilian institutions, experts and stakeholders. The OECD has selected these six dimensions, among other things, for the focus of the analysis. These should not be considered an exhaustive list of the multi-level governance challenges in Brazil; they can be revised and refined depending on the needs and evolution of the country and its audit institutions. For example, audit institutions may consider relevant the stakeholder engagement dimension more in-depth.
Why is it important to focus on these six dimensions in the generic MLG assessment framework?
1. Assignment of responsibilities
In OECD countries, responsibilities over a wide range of policy sectors are shared across levels of government. In many cases, regional responsibilities and functions (regulating, operating, financing and reporting functions) are shared with other institutional government levels, be they central or local. The extent of responsibility-sharing depends on the service in question. For example, responsibilities tend to be shared more often in public transport than in childcare or elderly care. Because subnational governments are embedded in national legislative frameworks, truly exclusive competencies rarely exist, even in federal countries. Countries share responsibilities for functional or financing reasons, through explicit legislation or through residual policy acquisition (Allain-Dupré, 2018[12]).
Due to the complexity of interactions in shared rule, there are many ambiguities in the assignment of responsibilities. In OECD countries, the unclear allocation of responsibilities and functions is particularly notable for policy areas such as infrastructure (transport), education, spatial planning, health and labour market policy; all of these often involve multiple tiers of government. Unclear assignment, when it is not driven by a precise objective, poses major obstacles in ensuring overall efficiency of public investments and local political accountability.
A principled and transparent division of powers is crucial for governments to deliver on their mandates and be held accountable by citizens. This is especially desirable for shared rule, i.e. when a function is the joint responsibility of several levels of government – as is often the case in the provision of education, health and social welfare due to their redistributive nature. These services are best provided locally when it comes to preference matching and tailoring of programmes to specific local needs, but at the same time higher-order legislative frameworks and financing may be required for equitable provision. Lack of clarity in the division of powers for concurrent/shared responsibilities contributes to government failures and/or inefficiency and inequity in public service provision.
Clarifying the allocation of responsibilities and functions is even more crucial in federal countries where responsibilities in a single sector can be shared between the federal, state and local government levels. Blurred definitions of these responsibilities can lead to misinterpretation and put more weight on the municipalities, especially in sectors such as education, health, land use, or climate actions. There is often a lack of clear definition in the legislation of the allocation of competencies in Brazil. This issue is an ongoing subject of discussion in Brazil; it was raised recently in debates over PEC 188/2019, which supervises the decentralisation of resources for states and municipalities. There is no general consensus as to how the relationship among governments should be redefined (Senado Noticias, 2019[13]).
2. Funding of subnational responsibilities
Local governments often have rigid budgetary arrangements and reduced financial room for manoeuvre. This is particularly relevant in Brazil since the new fiscal regime was approved by the 2016 Constitutional Amendment (EC 95/2016); the regime requires a 20-year period of fiscal consolidation, including in primary sectors (e.g. health and education). Indeed, tax and own-source revenue represented 34% of total municipal revenue in 2016 in Brazil, against 65% for transfers from the Union and state governments (compared to 44.5% and 37.2% respectively on average in the OECD area) (OECD, 2019[10]). This can create unfunded and underfunded higher-level mandates, undermine local accountability, and endanger service provision.
In federal countries in particular, regional disparities in spending across jurisdictions may be a warning sign of underfunding. In this regard, choices regarding subnational transfers, own revenues and borrowing should also be determined by the need for aligning policy objectives. Transparent, stable and regular intergovernmental fiscal arrangements tend to determine to a large extent subnational governments’ financial capacity to invest. In addition, the conditions for mobilising own-source revenue depend on the legal framework and the possibility for municipalities to create new taxes and fees, or their room for manoeuvre in acting on the tax rates or bases. In this category, multi-level governance criteria could be complemented by quantitative criteria on subnational finance (Frente Nacional de Prefeitos, 2020[14]). The Brazilian fiscal system is particularly rigid in this regard (Fernandes and Santana, 2018[15]). Most intergovernmental relations, particularly in relation to tax revenues, are detailed in the Federal Constitution, and therefore cannot be established or modified by the federal political or economic authorities without an amendment to the Constitution. The Federal Constitution defines the taxation responsibilities for each federative entity, and it establishes basic rules for the collection of subnational taxes. However, the Union tends to rely heavily on tax waivers and social contributions, which reduces the tax base and undermine fiscal autonomy at lower levels of government (OECD, 2019[16]).
3. Capacities of subnational governments and capacity building
The capacity of local governments varies greatly within countries, and the capacity gap between rural localities and large metropolitan areas can be substantial. Large regions, particularly those that are more autonomous and have numerous staffs, can tap into a diverse range of professional skills. The same is not necessarily true for small regions, municipalities, newly created regions, or where decentralisation reforms have outpaced improvements in administrative capacity. Larger municipalities, on the other hand, face complex tasks and might not have the workforce skills to address those tasks effectively. Defining, structuring, implementing, operating and monitoring public policies and programmes all require a very diverse set of capacities, which need to be assessed at every stage of the policy cycle.
In Brazil, the low level of capacities to design and implement policies at the Union, state and local levels is probably one of the most important bottlenecks for effective public service delivery. For example, Brazilian subnational governments often lack access to information on existing federal programmes: They do not have the sufficient capacity to understand how they could benefit from them, and lack the tools and knowledge to apply to these programmes or approach the federal authorities.
Building and strengthening subnational capacities in any country is a long-term commitment that requires sustained resources and political will from both subnational and central/federal government levels. This is particularly true in countries with a high rate of commissioners or consultants working in public administration – as is the case in Brazil (OECD, 2020[17])– hence the necessity to identify municipalities with the greatest capacity problems. This is a prerequisite for public and private institutions to be able to target capacity-building programmes to the municipalities that need it the most (OECD, 2013[18]).
Attention must also be paid to the capacity of municipalities to innovate and draw inspiration from other localities and international good practices. Knowledge sharing can take place through observatories inventorying good practices, or city networks, that can catalyse the development and implementation of policies at the local level by providing advice and guiding local governments. The exchange of good practices among subnational governments can also ensure that innovative solutions are adopted more widely.
4. Co-ordination among levels of government
Quality investment and service delivery require effective co-ordination arrangements to ensure that strategies and priorities at both the national and subnational levels are working in harmony. Co-ordination among levels of government is particularly necessary when it comes to shared and overlapping responsibilities, as is often the case in federal countries. Co-ordination is not limited to government entities; it can also extend to other stakeholders. It can help to align priorities and objectives and to overcome challenges posed by unclear assignment of responsibilities or by a series of information, financing, and capacity gaps that impede efficient use of resources. This complexity is exacerbated in Brazil, a country where coalition governments make co-ordination among ministries that much more difficult (OECD, 2013[18]). Both horizontal and vertical co-ordination are therefore crucial to limit the growth of inequalities and ensure coherent policies in unitary and federal countries. Through enhanced dialogue and exchange of information, well-managed co-ordination can also reinforce trust among the different levels of government and lead to more effective policy implementation.
Co-ordination mechanisms also need to be strengthened by the centre of government which has the capacity to steer policy development and implementation through a supporting and advisory role for subnational governments and non-governmental stakeholders. Effective policy co-ordination in fact calls for a whole-of-government approach, where the centre of government shares with other government actors some key responsibilities, such as supporting effective decision making; overseeing the quality of the policy process; facilitating policy co-ordination; communicating relevant policy messages; and monitoring and evaluating outcomes (OECD, 2018[19]). This changing role of the national government is a key trend in several OECD countries where national-level governments are increasingly playing a strategic role focused on setting objectives, co-ordinating policy and monitoring performance (OECD, 2019[3]; OECD, 2017[20]). The role of the centre of government in Brazil must be seen in the light of political situations and evolutions over recent years. As of 2017, Brazil is among the countries where senior management in the centre of government changes the most with a change of government. That can entail particular difficulty in ensuring the continuity of supervisory responsibilities over the long-term (OECD, 2018[19]).
Multi-level co-ordination can also enable better adjustment of policies to the needs of different localities. When decisions are taken in silos, resources are more likely to be assigned to policies that do not necessarily respond effectively to local needs. It is through joint actions that policies and service delivery may target the proper scale, internalising positive or negative spillovers and implementing the complementary measures needed to make the most of the interventions. Multi-year programming and policies can further support a co-ordinated approach and enable better forecasting and alignment of objectives across levels of government.
Horizontal co-ordination can also offer significant advantages in terms of policy delivery and efficiency of public investments. It can be fundamental in avoiding duplication of unsustainable investments at the subnational level, and in promoting economies of scale for investments and service delivery. Horizontal co-ordination can take the form of inter-municipal or inter-state co-operation, through which subnational governments can choose to join forces if this is the way to improve their efficiency and effectiveness (e.g. Arranjos de desenvolvimento da educação arrangements for education development in the education sector). For certain services for instance, efficient provision might be ensured when the service is provided by a group of municipalities covering a bigger territorial area than the municipality itself. Sometimes, legal requirements for services like waste management, water supply, transport and economic development need to be delivered through some form of inter-municipal co-operation (Council of Europe, 2010[21]).
While policy-makers recognise the advantages that it can bring, co-ordination is in general difficult to put into practice. Policy-makers from different sectors and levels of government tend to work in silos. Transaction costs and often a degree of competition for funds can be important barriers to co-ordination. It is not surprising that, for example, among the 15 dimensions of institutional quality for efficient public investment management listed by the International Monetary Fund, central-local co-ordination is the one where advanced economies tend to fare the worst (IMF, 2015[22]). Transaction costs, competitive pressures, resource constraints, differing priorities, and fears that the distribution of costs or benefits from co-operation will be one-sided can all impede efforts to bring governments together. National and subnational governments often recognise that co-ordination represents a major challenge for them.
5. Performance monitoring and transparency
Performance monitoring and evaluation (M&E) mechanisms are essential for clarifying the outcomes to be achieved and improving the efficiency and effectiveness of different policies. In the early stages of the policy cycle, ex ante appraisal can facilitate the prioritisation of projects and offer insights into the potential territorial impacts and benefits of spending decisions. Tools for ex ante appraisals/analyses are diverse; they may include for example cost-benefit analyses, environmental impact assessments, and territorial impact assessments. As to investment, ex ante assessments serve to assess long-term operational and maintenance costs from the early stages of the investment decision (Mizell and Allain-Dupré, 2013[23]). Subnational governments could require ex ante appraisals to encourage actors to produce high-quality/high-accuracy assessments when undertaking decision-taking processes.
Sound M&E can also help to identify challenges and obstacles that hinder effective policy implementation, as well as show the way forward to address the challenges, based on better appraisal and lessons learned about what has worked (or not) in the past. Governments should encourage the production of data at the right territorial scale to inform policies and produce evidence for decision making. To achieve such comprehensive assessments, it is also necessary that evaluation and monitoring criteria and mechanisms be defined early in the policy design process, and should not be limited to budget execution (OECD, 2018[24]).
Then focusing on performance through M&E mechanisms, including a clear indicators system, allows improving the efficiency and effectiveness of public policies, investment and service delivery. For instance, the municipality of Brusque in Brazil is aggregating information from the state education from the various budgets (OECD, 2019[16]). It does so by linking policy objectives with outcomes, revealing information throughout the policy cycle that should feed into decisions regarding policies in subsequent stages. Still, setting evaluation standards and using their results in future interventions are not always easy. Beyond the capacity needs involved, policy M&E entails additional costs that need to be balanced with the need to pursue effectiveness (OECD, 2019[4]).
6. Fiscal equalisation systems and regional policies to reduce territorial disparities
OECD countries face increasing territorial inequalities. This is particularly true in countries where regional governments have varying fiscal capacities and varying fiscal needs, and therefore varying ability to provide local public services. Among those countries is Brazil, where the level of regional inequalities, while decreasing in recent decades, is still among the highest in the world. These territorial disparities are due in part to municipalities having widely varying levels of resources, both quantitative and qualitative, to carry out their responsibilities (OECD, 2019[10]).
Equalisation systems, as part of a broader fiscal framework, may contribute to reducing territorial inequalities. Fiscal equalisation aims to correct imbalances among subnational governments and thus foster equity among territories, be they regions or localities, rural or urban. There is a large array of equalisation mechanisms, both vertical and horizontal. Equalisation arrangements, if they are carefully designed, can sustain subnational governments most in need, while promoting the tax and development efforts of subnational governments of various types. They can also play a positive role in strengthening an internal common market and a unified economic, social and political union.
However, fiscal equalisation systems are often contested for their complexity, lack of transparency, and potentially negative incentive effects on tax base development in both wealthier and most-in-need territories. To offset these potential negative effects, fiscal equalisation policies need in particular to be accompanied by proactive regional development policies (OECD, 2019[3]). Most OECD countries have such policies in place to reduce territorial disparities, in particular between urban and rural areas. In Brazil, the National Policy for Regional Development (Política Nacional de Desenvolvimento Regional, PNDR), last updated by Decree no. 9.810 in 2019, is a legal instrument to guide the federal government’s action towards reducing economic and social inequalities across the regions, and promoting more territorially balanced growth. Its implementation remains limited, however, and the policy has until now failed to meet its objectives, partly due to a lack of political consensus (Santos, Bessa and Lucio, 2017[25]).
Define maturity indicators for each dimension
For each of the above dimensions, the OECD has identified several maturity indicators that audit authorities should consider when assessing the multi-level governance performance of state and municipalities in Brazil. For each indicator, a detailed description is provided as a reference of “good practice” as well as the rationale for choosing it; also included are a description of the indicator, its main characteristics, and its relation to multi-level governance issues. Each maturity indicator also details the sources of information for the TCs to assess the state or municipality situation and so be able to identify the level of maturity of each criterion. The model specifies a three-level maturity scale associated with different colours: Red for low, yellow for medium, and green for satisfactory level (see below).
This generic assessment framework on multi-level governance can serve as a guide for TCs to develop their own document, to orient audit of multi-level governance in decentralised policy sectors. It provides a guide to the main issues that should be assessed in depth to evaluate the degree to which state and municipal governments in Brazil are equipped to design, implement and evaluate decentralised policy responsibilities and functions. It is important to consider that this tool is flexible in terms of applicability and can, with adjustments, be used at the state and municipal government levels. It can also be adjusted to fit with various sectors of decentralised policies (education, health, etc.).
The different maturity levels were defined based on the 10 guidelines for making decentralisation work (OECD, 2019[3]) and the OECD Recommendation on Effective Public Investment across Levels of Government (OECD, 2019[2]).
A summary of the maturity indicators included in the framework can be seen in Table 3.2 and the comprehensive generic assessment framework, with an explanation of the rationale, the unit of analysis, the sources of information and the three maturity levels, can be found in Annex 3.A.
This section illustrates how Brazilian audit authorities could apply the methodology described in the previous section for Steps 1-3 to a particular policy sector, in this case, education. For this, to illustrate step 1 Describe the MLG system for a policy area the first subsection provides a general overview of the main challenges of the multi-level governance system in education policies in Brazil. The second subsection provides a more detailed analysis of the selected six dimensions. Each subsection details how the OECD proceed to elaborate the general overview and the selection of the six dimensions. This analysis, was also developed taking the generic framework developed in the previous section as a reference. Based on this, the Annex details examples of maturity indicators that audit institutions may use to audit education policies in Brazil.
It is important to recall that the implementation of Steps1-3 of the model presented in this chapter is not a full and detailed assessment of how the MLG system works in the country for a specific policy sector. The main purpose is to provide a general description of how the system works in order to identify the main key challenges that deserve a deeper analysis. During the audit, audit institutions will be able to conduct a deeper analysis of the MLG system guided by the maturity indicators.
Describe the MLG system for a policy area
To describe the MLG system assessment framework for the education sector in Brazil, the OECD conducted a short literature review on the topic. Besides academic papers, the OECD referred to previous OECD studies where the MLG of the education sector was described and assessed.
Overview of the key challenges of the multi-level governance of education policies in Brazil
The multi-level governance of education policy is particularly relevant in Brazil where the education system is highly decentralised, with one federal district, 26 states and 5 570 municipalities, each with their own school system. The responsibility for the provision and oversight of public crèches and basic education is shared among the different levels of government. While responsibility for providing strategic direction and a substantial proportion of public funding lies with the federal government, constitutional responsibility for providing early child education and care (ECEC), primary and secondary education rests primarily with the states and municipalities. Municipalities are responsible for most of pre-school and fundamental education, while state governments are in charge of secondary education (OECD/UCLG, 2019[26]). Moreover, states and municipalities may establish their own goals and political strategies for education. The Ministry of Education is entitled to play an important role in setting overall education policy setting guidelines for the organisation of educational programmes, and organising national assessment systems. In this context, a key challenge for the audit process of education policy is the multiplicity and diversity of actors involved in governing and providing educational services to citizens in Brazil.
This high level of decentralisation and complexity results in considerable variation in teacher labour markets, curricula, levels of access to schooling and management systems, and local education plans (Carnoy et al., 2017[27]). Teachers’ career plans and salary grades, for instance, are set up at state and municipal levels. Since 2016, however, there has been a recentralisation of the national programme, with the release of a National Common Curricular Basis (BNCC), whereas before that, local schools each had their own curriculum.
Research has highlighted important disparities in quality between urban and rural schools, and between richer and poorer regions of Brazil identifying it as a key challenge of education policy (Bruns, Evans and Luque, 2011[28]); (Pereira and de Castro, 2019[29])). Municipalities vary greatly in size, internal organisation, resources and administrative capacity with a direct impact on the performance of the education sector. When considering the Municipal Human Development Index (IDHM) for Education, more than 50% of municipalities in the South and Southeast regions registered medium or high levels. In the North and Northeast regions, however, 90% of municipalities were ranked in the very low or low levels of the Index (PNUD, 2016[30]). Furthermore, in urban areas all states lie at the medium or high level of IDHM Education, whereas in rural areas all states presented very low or low levels (Pereira and de Castro, 2019[29]). Another noteworthy data is Brazil's aggregate illiteracy rate, which was 10.2% in 2010, but with 7.54% of illiteracy in urban areas and 24.64% in rural areas (Pereira and de Castro, 2019[29]). The current multi-level governance framework of education addresses some of these challenges, through financial and technical assistance to municipalities the most in need. However, key challenges remain for several municipalities, in particular those in rural areas and in the north and northeast regions.
Identify the key elements that should be further assessed
To identify the key elements of the education sector that an audit might focus on, the OECD used different means, among which:
The literature review used to describe the MLG system
Discussions with key experts on the education sector in Brazil and revision of presentations made in conferences on the topic
Discussions and inputs from the TCU and TCs with experience in auditing education policies
Review of key fiscal data
The generic assessment framework developed by the OECD (see Annex 3.A)
Resulting from this process, the OECD identified six key elements that should be assessed more in detail during the audit of education policy at the municipal level. These six dimensions are described below.
Based on this exploratory exercise, the OECD has then developed the assessment framework by designing maturity indicators. The full table with the maturity indicators can be found in Annex 3.A. As a dynamic process, the TCU and TCs may consider adapting this framework if they consider it appropriate.
Dimension 1: The assignment of responsibilities in the education sector
In Brazil, as in many other federal countries, education is a shared responsibility among the municipalities, the states and the federal government. It is set in the Brazilian Constitution (1988) and the Law of Directives and Bases of National Education (LDB) (Law nº 9.394/1996). Municipalities are responsible for primary and lower secondary education, whereas state governments have the competency to provide both lower and upper secondary education. The Union is in charge of the management of higher education and vocational training. The LDB commands states to prioritise high schools and leave the lower secondary education level to municipalities, under their guidance (Castro, M. H. G., 2010[31]); (Bruns, Evans and Luque, 2011[28])).
This shared-responsibility for secondary education creates important challenges of duplication and fragmentation in education delivery, as it is the case in several OECD countries where responsibilities are shared. It also fostered a spirit of competition among states and municipalities (TPE, 2018[32]). For example, it is common that state schools and municipal schools of the same city adopt different school calendars, so that classes do not start on the same dates and that vacation periods do not match. Another noticeable difficulty is transportation: either there are two separated transportation services in the same city, which proves to be an inefficient way to allocate resources, especially in small cities, or municipalities have to bear the financial burden of providing transportation services for both municipal and state schools (Diniz, 2012[33]). The intervention of the federal state was necessary to alleviate this problem by assuming the costs of school transportation.
The three levels of government operate under a constitutionally mandated “collaboration regime”, in which education is a shared and concurrent competency (art. 211 of the Constitution) (Dourado L. F., 2013[34]). This regime, however, fails to ensure a clear and detailed distribution of competences (TPE, 2018[32]), and the situation persists where municipalities tend to have high levels of responsibility but low levels of own-source revenues and little control over the allocation of transferred resources (Diniz, 2012[33]).
Regarding the distribution of functions, the federal government has at least three main functions related to co-ordination of education policies: legislative; planning and evaluation; and redistributive and steering (TPE, 2018[32]), as per article 8 of the Law of Directives and Bases of National Education (Law nº 9.394/1996). Nonetheless, the competency to legislate about education is concurrent among the three levels of government, each at their sphere of action (art. 24, IX of the Constitution).
The legislative function of the federal government refers to enacting laws and regulations, for instance the Law of Directives and Bases of National Education in 1996 and the Law of the National Education Plan in 2014, just to cite a few landmark ones. According to article 22 of the Constitution, it is the exclusive responsibility of the federal government to enact the guidelines and principles of national education (Dourado L. F., 2013[34]). The federal government also edits deliberations and resolutions through collegiate bodies such as the National Education Council (CNE). It is in charge of designing country-wide programmes, e.g. the programme of the National Common Curricular Basis (BNCC) and the National Programme of School Meals (PNAE), among many others.
The planning and evaluation functions of the federal government comprise editing plans, for instance the National Education Plan (PNE), and managing the national system of information and evaluation. The federal government compiles information about school functioning and budgeting, as well as students’ performance. Among the standardised evaluations regularly applied, the Basic Education Evaluation System (SAEB) was introduced as of 2019 to replace the Aneb and Anresc system (ANA), also known as Prova Brasil. Departing from the results of these two tests, and combining them with students’ approval rates, the government has developed an index called Basic Education Development Index (IDEB).
Lastly, the redistributive and steering function of the federal government involves providing technical and financial assistance to local governments. The federal government does so mainly through programmes of direct financial support to schools, such as the National Programme of School Meals (PNAE), the National Programme to Support School Transportation (PNATE) and the Direct Money at School Programme (PDDE). The purpose of the latter programme is to provide financial assistance to public schools with the aim of improving physical and pedagogical infrastructure and management practices (Diniz, 2012[33]).
At the subnational level, states and municipalities are in charge of developing and enacting their own laws and plans, including the budget and the local education plans. As will be discussed in the sections below, subnational governments are obliged by a somewhat rigid financing structure, with constitutionally-mandated minimum levels of investment. When it comes to planning, because the federal government provides the general framework upon which local education systems can function, local plans tend to replicate the principles and objectives enacted by the federal government. Still, they often adopt their own action plans and set their own targets, meaning that, when it comes to implementation, they have to be more action-oriented and proactive. Subnational governments are free to develop their own evaluation systems, and many of them have done so, but the overall trend is to assist the execution of nationally-designed evaluation systems, and in exchange to receive detailed data, disaggregated at school level.
Dimension 2: Funding responsibilities in the education sector
The Constitution mandates a minimum level of investment in education (art. 212 of Constitution and art. 69 of Law nº 9.394/1996). States and municipalities have to contribute at least 25% of all their tax revenues to the maintenance and development of education, including the ones coming from transfers, and the federal government, 18%. This minimum level of 25% for subnational governments includes own-source tax revenues and revenues that originate from transfers of the federal government (Tesouro Nacional, n.d.[35]). The concept of “maintenance and development of education” is defined by Law (arts. 70 and 71 of Law nº 9.394/96). It comprises spending on teacher salaries, school operation costs, school transportation programmes, scholarships and educational materials. It does not include social assistance programmes, infrastructure works, subventions to philanthropic entities, or research conducted outside of the school system.
The mandatory minimum level of investment in education varies across regions. This standard is stricter than the minimum level of investment mandated in the Constitution, as the reference is the cost to deliver quality education to every student. Taking the index Cost Pupil Quality (CAQ) as a parameter, it is estimated that, in 2019, 43% of the 5 570 municipalities and five states (Minas Gerais, Paráiba, Maranhão, Pará e Amazonas) invested less than the satisfactory minimum to promote quality education (TPE, 2018[32]). The mandatory minimum level of investment varies across regions, which have large disparities in terms of tax revenue. As a result, if this was the only financing rule in place, it is estimated that spending in the northeast region would amount to less than R$ 100 per primary student per year in municipal schools, below the average spending of Bolivia and Nicaragua, whereas in the southeast region this spending could go up to BRL 1 500 per primary student per year, around the same levels of Korea and Singapore (Bruns, Evans and Luque, 2011[28]).In addition, in some cases, funding for education and from the FUNDEB, may also be diverted to cover other types of expenditure. This is the case, for example, of the state of Sao Paulo where resources from the Fundeb have been used to pay retirement pensions in the state of Sao Paulo (OECD, 2020[17]).
To address these regional imbalances, the federal government adopted two parallel courses of action. One was setting a minimum level of spending per student in primary education. The other was creating an equalisation fund, called Fund for the Development of Primary Education and Appreciation of Teachers (FUNDEF), later transformed in 2007 into the Fund for the Development of Primary Education and Appreciation of Teachers (FUNDEB).
Another important source of revenues for education is the educational allowance (Salário-educação). It is a social contribution extracted from 2.5% of companies’ monthly payroll. It was created in 1964 and since then has undergone several reforms. Today, as set in the Law nº 10.832/2003, 10% of the net revenue of the educational allowance goes to the National Fund for Education Development (FNDE), and 90% is automatically distributed among the three entities, in the proportion of 1/3 to the federal government and 2/3 to state and municipal governments. The 2/3 is shared according to enrolment numbers, with the objective of financing education programmes, projects and actions (art. 212, §6º of the Constitution).
In addition, states and municipalities receive legal transfers from the federal government, in virtue of universal distribution programme (Tanno, 2017[36]). The only criterion for distribution is the number of students enrolled in each level of schooling. It is not associated with the entity’s budget nor with any equalisation mechanism. Moreover, no counterpart is demanded (Diniz, 2012[33]). The programmes are the National Programme of School Meals (PNAE), the National Programme to Support School Transportation (PNATE), the Direct Money at School Programme (PDDE) and the National Programme of Textbooks (PNLD).
Table 3.3 summarises the funding structure of education in Brazil, per type of funding mechanism and per level of government. The structure has not suffered substantial alterations since 2010.
There is little room for variation across subnational governments
This relatively fixed funding structure in the education sector in Brazil leaves little room for subnational governments to act on their revenue streams. There are, however, examples of subnational governments adopting differentiated approaches, for instance by spending above the constitutional minimum or by altering their own taxes, to collect more revenues for education.
Spending levels above the minimum level is fairly common among subnational governments in Brazil. In 2017, all states spent above the minimum of 25% (Figure 3.4). The level of spending does not relate to regional inequalities, though. As it reflects a percentage of the budget, and since the budget of richer states is higher than the one of poorer states, states with similar percentages would actually have spent disparate amounts of resources on education. The graph below shows the percentage of education expenditure in the year 2017 at the state level, using data collected by the Ministry of Education – FNDE via the system SIOPE.
There are also cases of poor municipalities spending significantly more on education, and doing so efficiently, which translates into higher student performance, in comparison to municipalities with similar income levels. Measuring the efficiency of public spending on education in 57 municipalities in the state of Alagoas, Wilbert and D’Abreu, a recent research found that municipalities with low Gross Domestic Product (GDP) per capita and low value spent per student were the most efficient. This might reflect that the amount of investment does not necessarily affect the quality of education (Matias et al., 2018[37]). In these same lines, analysing municipalities of the state of São Paulo, higher spending did not translate into better educational performance when it was not associated with an increase in the efficiency of spending. On the other hand, a study of the 26 state capitals – hence of large cities – found no significant relationship between efficiency in spending and better performance in primary education (Matias et al., 2018[37]).
Subnational governments may also channel additional revenue to education by altering the calculation and collection formula of a given tax, and combine it with a distribution mechanism. The state of Ceará has adopted such a strategy. The state established that 72% of revenues collected from the Value-Added Tax (ICMS) would have to be spent on education. The state did not augment the tax rate, but simply conditioned this percentage of mandatory transfers to municipalities, resorting to the Index of Quality of Education (IQE) as a reference (Codes, A, 2018[38]). This policy has enabled small municipalities in the state of Ceará with a good record of educational performance to receive more transfers. For some of these municipalities, the share of ICMS transfers has even surpassed the one of FUNDEB transfers, since these are based on enrolment numbers (Codes, A, 2018[38]). Some states are starting to follow this example of Ceará, for instance in the state of São Paulo where a law was proposed (Assembleia legislativa do Estado do Sao Paulo, n.d.[39]).
A low spending efficiency in the education sector
Despite the high share of public spending with respect to GDP in education in Brazil, this spending does not directly translate into better educational results. While compared to OECD countries, Brazil spends a relatively high proportion of national wealth, although the country’s comparatively low GDP per capita and young population mean that this translates into comparatively low spending per student. Brazil has performed poorly at the PISA evaluation in the past years. The country spends around the same than Mexico and more than Turkey and Thailand on education, yet its student performance results are worse (Oliveira et al., 2015[40]).
This can be explained by the poor efficiency of resource allocation, which has a lower impact than the amount spent (Oliveira et al., 2015[40]); (Glewwe et al., 2013[41]); (Hyman, 2017[42])). For sure, a certain standard of investment is fundamental: Brazil has seen an important progress in enrolment rates and educational attainment levels since the early 1990s precisely because the amount of resources invested in education increased significantly during this period (Bruns, Evans and Luque, 2011[28]). Yet making progress from there means spending better, not simply adding more resources (Oliveira et al., 2015[40]). It is estimated that more efficient spending could reduce costs at the municipal level from 3% to 30%, depending on the level of efficiency reached (Moreira, 2017[43]).
The allocation of spending across education levels is considered as one of the key factors that hinder better results at the current level of education spending in Brazil (Bruns, Evans and Luque, 2011[28]), along with persistent high repetition rates and high costs per graduate; rising teacher costs; little cost-effectiveness research; and corruption and mismanagement of education funds. Indeed, while every OECD country spends more per student on tertiary education than at the primary level – on average, twice as much – the ratio in Brazil is extreme: almost six times as much. The issue is not the share of GDP Brazil devotes to tertiary education, as 0.75% is well below the OECD average of 1.5%. The issue is the very small number of students in public higher education relative to spending.
Dimension 3: Co-ordination mechanisms in the education sector
The collaboration regime foreseen in the Brazilian Constitution conveys a decentralised and complex system of allocation of responsibilities in the education sector. Yet, there are very few institutional mechanisms to induce or command systematic and solid collaboration among the three levels of government in this sector (Furtado and Soares, 2018[44]). Indeed, the collaboration regime, even if constitutionally established, has not yet been regulated by specific laws. A National Education System (SNE), as foreseen in the Law of the National Education Plan (art. 13 of Law nº 13.005/2014), could promote such mechanisms and create others. The SNE has however remained on the policy agenda for several years, without being effectively implemented. According to researchers, advocacy groups and teachers’ unions, the creation of the SNE could clarify responsibilities, improve co-operation and strengthen the federal roles of social redistribution and technical assistance (Dourado L. F., 2013[34]) (TPE, 2018[32]).
The creation of a National Education System has been on the policy agenda for some years now. It bears the promise of fulfilling the principle of the “collaboration regime” (Dourado L. F., 2013[34]) which has not yet been regulated. The proposals’ document for the 2014 National Education Conference (Conferência Nacional da Educação – CONAE), elaborated by the National Education Forum (FNE), had argued that the creation of a National System would contribute to achieving the following objectives:
To promote common educational guidelines throughout the national territory, with the perspective of overcoming regional inequalities and promoting the right to quality education;
To define and guarantee common educational purposes, guidelines and strategies, without prejudice to the specificities of each system;
To reinforce the federal government’s role in promoting articulation, standardisation, co-ordination and regulation of public and private national education.
Later in 2014, with the Law of the National Education Plan (Law nº 13.005/2014), the creation of such a system became mandatory. Article 13 of the Law states that:
Art. 13. The public power must establish, in a specific law, two years after the publication of this Law, the National Education System, responsible for the articulation between the education systems, in a collaborative regime, for the effectiveness of the guidelines, goals and strategies of the National Education Plan.
Given that the National Education System would have to be created by a specific national law, advocacy groups and non-governmental organisations, together with congressmen and senators whose platform of work is education, have been promoting this agenda in the National Congress. The Complementary Law Project nº 25, which foresees the creation of the SNE, was proposed in February of 2019 at the Chamber of Deputies. However, it awaits appreciation by the Education Commission since March of that same year. Another Law Project enacting the SNE was proposed in 2019, this time at the Federal Senate. The Complementary Law Project nº 235, proposed in October of 2019, awaits appreciation by the Education Commission since December of that year.
A specific law of the National Education System, according to the non-governmental organisation Todos pela Educação (2018) and the National Campaign for the Right to Education, should bring the following changes:
Establish the competences and duties of each entity more clearly, with emphasis on strengthening the federal government’s role of co-ordinating national education, while diminishing its role as executor of policies.
Establish national benchmarks for the provision of quality basic education, through the adoption of the index of Cost Pupil Quality (CAQ), and with the support of a tripartite council or body.
Assign to each state government the role of overseeing basic education policies in their territories, by establishing clear criteria for issues such as: enrolment, curriculum, evaluation, teaching materials, selection and training of teachers.
Create mechanisms to foster collaborative practices between municipalities, among which inter-municipal consortia and the institutionalisation of Education Development Arrangements (ADE).
Give special attention to the North and Northeast regions: in light of the Brazilian federative pact, the SNE must guarantee the necessary conditions for quality and equity in educational care, especially at the basic level, seeking to overcome historical regional inequalities.
Source: Adapted from: (Dourado L. F., 2013[34]; National Education Conference (CONAE), 2013[45]; TPE, 2018[32]; Campanha Nacional pelo Direito à Educaçao, n.d.[46]; Camara dos Deputados, n.d.[47]).
Historically, the Ministry of Education (MEC) has enacted mostly top-down policies, based on general assumptions about the needs and deficiencies of subnational governments. This strategy has paid off in the 1990s, resulting in an increase in school attendance and years of schooling. As the policy focus is shifting from access to education to quality of education, the federal government must face the challenge of addressing the remaining gap in income equality and educational achievement across regions. Top-down, centralised policies might not be able to account for the heterogeneity of Brazilian cities and states. The National Education Council (CNE, 2012[48]) has stressed that, despite the constitutional design of a non-hierarchical federalism, the federal government is still anchored in a vertical and centralised mode of operation. It is thus urgent to invest in territorial policies and in collaborative approaches to policy-making (CNE, 2012[48]), from three perspectives: co-ordination between the Union and municipalities; states and municipalities; and horizontal co-ordination.
Co-ordination between the federal government and municipalities
Given its steering and planning functions in education policy, the federal government ought to provide a framework for co-operation. In the institutional arena, this steering role was reflected in the Secretary of Articulation of the Educational System (SASE) of the Ministry of Education (MEC). The Secretary had, among others, the attributions of contributing to the creation of a national education system; fostering cooperation between entities; and supporting subnational governments in the elaboration, implementation and evaluation of their Education Plans (article 30 of Decree nº 7.690/2012). Although it had considered it as a positive institutional measure towards a more effective and solid collaboration regime (CNE, 2012[48]), it was extinct in 2016.
Moreover, the federal government has created and strengthened several participation and agreement spheres, such as conferences, forums, collegiate entities and deliberative councils, among others. These mechanisms often include subnational governments, as well as the participation of the civil society, unions and universities. Some of these joint bodies have decision-making powers, while others function as permanent dialogue instances that agree on common priorities and establish guidelines. They produce documents that form the basis of new policies, such as the proposals of the National Education Conference (CONAE). Table 2 provides examples of such arenas of participation and consensus-building.
A key collaboration mechanism is the Co-ordinated Action Plan (Plano de Ações Articuladas – PAR), a strategic participatory planning tool (Fermandes and Nogueira, 2018[49]). Departing from a diagnostic of the educational scenario, municipalities and states develop a 4-year plan, with strategies to address the identified challenges. Through a participatory process based on questionnaires and joint meetings, municipalities and states identify their greatest needs and set actions to address them. Once the PAR is formulated, the subnational entity presents it to the Ministry of Education (MEC), with whom it signs a letter of agreement. Departing from the actions proposed in the PAR, the Ministry of Education may provide adequate technical and financial assistance. This is critical because the federal government has typically played a timid role in providing technical assistance (TPE, 2018[32]).
The PAR set a new standard of relationship between the federal government and the other entities. At the beginning, some states and municipalities did not know what to demand, and the federal government, in turn, did not know what to offer (Fermandes and Nogueira, 2018[49]). Most states and municipalities elaborate their PAR at every 4-year cycle since 2007, when it was created. Although not mandatory, according to the Resolution CD/FNDE nº 14/2012, the elaboration of such action plans is a condition for obtaining technical and financial assistance from MEC. In other words, the PAR is a prerequisite for subnational entities to receive technical and financial assistance from the federal government, which will be paid with resources of the National Fund for Education Development (FNDE). Thus, the strategic planning process of PAR has allowed for qualification of both demand and supply of educational programmes.
Co-ordination between states and municipalities
The relationship between states and municipalities in Brazil regarding education differs from state to state. In some states, more than the institutional set-up in itself, the definition of rules and mechanisms for collaboration between the entities often depends on the negotiating capacity of educational managers (Fermandes and Nogueira, 2018[49]). Overall, the co-ordination mechanisms between states and municipalities can be divided into five categories (Table 3.5). Departing from this typology, different policies have been identified. They range from joint programmes of school enrolment to continuous training given to teachers and local civil servants.
Most state governments have developed transportation and school meal programmes for municipalities. Some states have created joint programmes regarding school calendar, enrolment systems and human capital management, such as Goiás and Acre (Segatto, 2015[50]). In others, it is more about assistance, guidance and information sharing.
In states such as Paraíba, state and municipalities have independent policies, with no significant degree of collaboration (Abrucio, L. F, 2017[51]). In the state of Pará the relationship is best characterised as conflictive: there are no joint programmes, negotiation instances or dialogue channels between the State Secretary of Education and the Municipal Secretary of Education (Segatto, 2015[50]).
Box 6 presents the trajectory, operation and results of the collaboration regime between the state of Ceará and its 184 municipalities, focusing on the Literacy at the Right Age Programme (PAIC), internationally praised as successful (Abrucio, Pereira and Seggato, 2017[52]; Carnoy et al., 2017[27]).
The Literacy at the Right Age Programme (PAIC) was launched in 2007 and institutionalised through the Law nº 14.026/2007, with the aim of ensuring that all children were literate by the age of seven. PAIC focuses on improving municipal and school management, directing them to children's learning. Since the programme's inception, the Ceará’s Secretary of Education (SEDUC) emphasised that partisan political issues should not interfere with education and has not treated allied mayors or state government opponents differently.
The PAIC was joined by all 184 municipalities in Ceará, through a cooperation agreement. By joining PAIC, the municipalities committed themselves to structuring teams in the Municipal Education Departments to implement the programme.
SEDUC itself underwent an organisational restructuring. The Co-ordination of Cooperation with Municipalities (COPEM) was created, with the objective of establishing a cooperation system to define policies and structure actions to raise the quality of early childhood education. In addition, an intermediary level between states and municipalities was enacted, composed of Regional Education Development Co-ordinators (CREDEs).
The programme has been reformulated over the years, undergoing incremental adjustments and changes. In 2008, financial support to municipalities was initiated through scholarships for the civil servants responsible for implementing the programme, to complement their salary.
In 2009, a financial induction mechanism to make literacy a priority was enacted, by changing the quota of the Value Added Tax (ICMS) redistributed to municipalities. Out of the 25% share that municipalities receive, 18% became directed to education, with basis on the Index of Quality Education (IQE). The IQE is calculated according to proficiency levels so that municipalities with the highest number of children at the appropriate level are valued, whereas those with children at the lowest levels of proficiency are penalised. The calculation also considers the percentage of students who participated in the assessments and the improvement rate over the previous year.
Another mechanism of financial induction adopted in Ceará is the Grade-A School Award, instituted in 2009 by Law nº 14.371. The award is destined to the 150 public schools that obtain the best literacy results measured by the School Performance Index - Literacy. The winning school first receives 75% of the total, but the remaining 25% is received upon the celebration of a technical and pedagogical cooperation agreement with one of the 150 schools with the lowest results.
The success of this policy depended on a well-rounded governance framework. The leadership was capable of continuous learning and changing, and of articulating and negotiating with other governmental and non-governmental actors to seek support. Collaboration was built around clear projects, defined in a dialogical way. Short, medium, and long-term goals were put in place. Adequate structures and incentives were set to municipalities, with a virtuous equilibrium between cooperation and competition. For teaching, a tripod was set up, based on the combination of training, adequate pedagogical material and monitoring and evaluation, in a feedback loop.
Source: (Abrucio, Pereira and Seggato, 2017[52]).
Horizontal co-ordination
There are very few institutional spheres for horizontal co-ordination amongst states or municipalities in Brazil. Indeed, cooperation between municipalities is reported to be low (Abrucio, L. F, 2017[51]). The most relevant mechanism is inter-municipal consortia, notably the Education Development Arrangements (ADEs) (CNE, 2012[53]). These arrangements hold great innovative potential to solving complex and multifaceted problems, while their magnitude remains small.
The National Council of Education defines ADE as a horizontal form of territorial collaboration aimed at ensuring quality education (CNE, 2012[53]). It is a networking model in which municipalities with geographical proximity and similar socioeconomic characteristics will exchange experiences and jointly address challenges in the field of education (Abrucio, L. F, 2017[51]). It is typically formed by municipal governments, but non-governmental organisations and non-for-profit private entities may participate, too (CNE, 2012[53]).
One of the objectives of ADEs is to develop a shared methodology among municipalities to increase the efficiency of actions and to measure the results of these actions with the aid of indicators (CNE, 2012[53]). It informs a model of evidence-based decision-making, departing from a common understanding of a shared local context.
In this sense, ADEs can adopt many different courses of action. They may comprise, for instance, the development of local education plans and local curricula. They may focus on training teachers and civil servants and establishing an arrangement of mutual transfers of civil servants between municipalities. A joint evaluation system may be put in place, and management styles might be assessed and improved. Joint programmes of service provision may be enacted, such as one concerning school transportation.
Between 2009 and 2017, 15 different ADEs were developed across the country (Abrucio, L. F, 2017[51]). Some remain active, while others have been desactivated. The ADEs of Chapada Diamantina, Florianópolis metropolitan region, Carajás Railroad and São Paulo Northwest are regarded by the specialised literature as successful (Abrucio, L. F, 2017[51]). In all of them, comprehensive assessments were conducted, culminating with the elaboration of joint action plans, and a collaborative network of educational management has been solidified. In some of them, non-state actors are partners, and they contribute with technical assistance and funding. Table 3.6 inventories the experience of these ADEs.
Since the creation of the ADEs, education results have improved in the participating municipalities. In the 12 municipalities of the ADE Chapada Diamantina, the Basic Education Development Index (IDEB) has doubled from 2005 to 2015 (Abrucio, L. F, 2017[51]). These municipalities had, as of then, the best IDEB of the whole state of Bahia. Out of the 20 municipalities of the ADE Carajás Railroad, 10 of them saw an increase of at least one point in the IDEB index between 2007 and 2015, on a scale that ranges from 0 to 10 (Abrucio, L. F, 2017[51]).
The main challenges faced by ADEs concern their long-term financial sustainability, especially in the poorer regions, the high costs of hiring and training qualified technical teams, the lack of engagement from civil society and weak oversight and support from the federal government (Abrucio, L. F, 2017[51]).
In expanding the model of ADEs, there are other actors that could be more strategically involved (Abrucio, L. F, 2017[51]). The Associations of Municipalities that exist in every state could advocate the interests of members in the creation of such collaborative arrangements and negotiate support from the state and federal levels. Non-for-profit foundations and NGOs of the sector of education are building a broad platform to advocate for policy reforms, promote good practices and support local managers. Universities, especially the ones located in regional hubs outside capital cities, could provide technical advice and develop evaluation methodologies for municipalities.
Chiefs of State Education Secretaries and Municipal Education Secretaries face common challenges and demands. With the goal of reuniting these chiefs, collegiate bodies have been created, such as the National Council of State Education Secretaries (CONSED) and the National Union of Municipal Education Managers (UNDIME). They function as spheres to share information and good practices and to align policy priorities. Through these forums or councils, states or municipalities can speak to the federal government as a unified voice (CNE, 2012[48]). As a consequence, they may have a stronger standing on what should be endorsed by the federal government.
Lastly, very few inter-school mechanisms of co-ordination have been observed. There is room for schools of different municipalities within the same state to adopt practices of peer learning, joint decision-making or joint resource allocation. Taking state policies as a departure point, they could improve the quality and efficiency of school management, by choosing to face similar problems together.
Some states have stepped in to make that collaboration amongst municipalities happen. In São Paulo, the 16 Regional Education Boards (DREs) act as an intermediate level between the state office and the municipalities (Barros, 2018[54]). They promote training programmes that gather teachers and administrators of the same educational region. In so doing, they facilitate information sharing and alignment of management and pedagogical practices. A common identity could be enhanced among municipal schools, promoting political alignment. Lastly, DREs could contribute to the development of an evidence-based management culture in municipal schools (Barros, 2018[54]).
Dimension 4: Capacities of subnational governments in the education sector
Capacity-building and technical assistance
States play a key role in strengthening the capacities of municipal administrations. They are closer to municipalities than the federal government and they are responsible for offering a sound framework of laws and programmes upon which municipalities can structure their education policy. They may step in to develop a joint school calendar, a joint registration system and even a joint curricular structure with municipalities. Moreover, states may offer technical assistance programmes to support the development of planning tools and management systems in municipalities.
Technical assistance programmes are an integral part of co-ordination mechanisms put in place between the federal, state and municipal governments (Table 3.7). These programmes often aim to support local governments in the design of local education plans, curricula and management systems. This is the case for instance in Ceará, Tocantins and Minas Gerais (Segatto, 2015[50]). Training programmes for municipal teachers and civil servants of the local education offices also exist in Piauí and Mato Grosso, too, besides the already cited Ceará, Tocantins and Minas Gerais. Nonetheless, in other states, this advisory role is less pronounced, as in Bahia and Sergipe (Abrucio, L. F, 2017[51]).
While the federal governments have typically played a rather timid role in providing technical assistance to municipalities (TPE, 2018[32]), the Co-ordinated Action Plan (Plano de Ações Articuladas – PAR) have led the way towards more co-operation in this field. Based on these plans, the Ministry of Education may provide adequate technical and financial assistance. States may as well resort to their evaluation systems to assess the performance of students in municipal schools. The adoption or not of these programmes reflects the level of co-ordination between states and municipalities, which may range from conflict and independent policies to joint programmes and close cooperation.
Horizontal co-ordination may also spur capacity-building. Education Development Arrangements (ADE) in Brazil are organised based on a networking model gathering a large number of municipalities. This allows for the exchange of experiences and providing peer-to-peer advice to other municipality in the field of education. They may comprise, for instance, the development of local education plans and local curricula. They may focus on training teachers and civil servants and establishing an arrangement of mutual transfers of civil servants between municipalities.
Capacity-building also takes place to strengthen municipal capacities in performance evaluation. The National Institute for Educational Studies and Research (INEP) provides technical assistance for subnational governments to create their own evaluation programmes. There are great advantages in the development of local assessment programmes: data breakdown at school level, collection of information of special interest and overall a deeper understanding of their specific context (Castro, 2002[55]). State schools in Ceará, for instance, use the results of mid-year standardised testing to adjust course content and thus increase student performance levels by the end of the school year (Codes, A, 2018[38]).
Strengthening capacities of education stakeholders
At the local level, the main challenges concern higher salaries, better working conditions, professionalisation of the school administration, with improved human resources management practices and regular all-staff meetings, and continuous training programmes designed to improve pedagogical methodologies (TPE, 2018[32]); (Matijiascic, 2017[56]); (Bruns, Evans and Luque, 2011[28])). Raising teacher quality is one of the main challenges for Brazilian education (Bruns, Evans and Luque, 2011[28]).
Raising teacher quality is one of the main challenges for Brazilian education (Bruns, Evans and Luque, 2011[28]). The career plan of the profession is not well-structured, the salaries are low (to compensate for the low salaries, 37% of teachers in Brazil work in two or more schools), many teachers are not specialised or do not work in their field of specialisation, and human resources management practices can be considered weak (TPE, 2018[32]). In all, the attractiveness of the teaching profession is low. Brazilian teachers are recruited from the bottom third of high school students, whereas in Singapore, Korea and Finland they come from the top third (Bruns, Evans and Luque, 2011[28]). Although Bruns and colleagues (2011) mention the rising costs of teacher salaries, more recent evidence shows that the average salary of teachers in Brazil is low (Matijiascic, 2017[56]). It lies below the average of university-educated professionals and there are important regional variations. In order to compensate for the low salaries, 37% of teachers in Brazil work in two or more schools, which in the end translates into a higher income level (TPE, 2018[32]). As a result, the average income of teachers lies above the national average. This is not very telling, since the average income of workers in Brazil is low, due in part to the high levels of informality of the economy and the low percentage of university-educated workers (Matijiascic, 2017[56]).
Moreover, there is evidence showing that the working conditions and the career path do not reflect an adequate environment for teachers' development. Basic matters of school infrastructure, security and psychological assistance are not provided. The career path rewards more those who get higher degrees than those who specialise in teaching methods, and time spent in the classroom does not award wage bonuses (Bruns, Evans and Luque, 2011[28]).
The capacity level of school administrators has also been identified as a relevant factor, capable of explaining performance variations among schools with similar socioeconomic contexts. School administration in Brazil is reported to be quite bureaucratic, overlooking pedagogical practices (TPE, 2018[32]). Research shows that 74% of principals of municipal schools are allocated in reason of political affiliation only, and that just 36% of principals pay consideration to aspects related to students' learning (TPE, 2018[32]).
In parallel, there is a lack of standardised hiring mechanisms for teachers across the subnational level; which reflects the decentralised educational system of the country. States and municipalities may conduct selection processes catered to their needs and conveniences. This also opens the door for arbitrary selection procedures, which mostly occur in small municipalities that may organise arbitrary selection procedures in which the best candidates is not selected. In many of them, selection is based on curriculum and interview, with no written exams. Aware of this challenge, the Ministry of Education (MEC) has mandated that every subnational entity establishes a formal recruiting process and a career plan for teachers (Bruns, Evans and Luque, 2011[28]). This should be organised, transparent entry exams, based on subject matter mastery and pedagogic credentials. Career plans must align performance requirements with salary increases. Small municipalities can pool together in order to organise a single, more rigorous selection process, more effective in attracting better candidates. The Regional Education Boards (DREs) or the co-ordinating bodies of Education Development Arrangements (ADE), where these exist, can help in setting up these procedures.
Teaching career paths may be improved by creating more competitive tracks and by associating gains with performance goals. The state of São Paulo, for instance, has adopted the Teacher Promotion Exam (Prova de Promoção) to create a highly paid career track for the top teachers in the public career. The rewards for passing this difficult test of content mastery are high: entering a new salary track in which the top-level salary amounts to four times annual per capita GDP, what would place teachers in the top 10% of professional salaries nationwide (Bruns, Evans and Luque, 2011[28]).
Recently, the federal government enacted two programmes with the goal of fostering the link between university students and the teaching career path. The Teaching Initiation Grants Programme (Pibid) offers teaching scholarships to university students who commit to exercise teaching in the public network after graduation. The Pedagogical Residency Programme offers paid internships to university students, with the aim of consolidating the importance of classroom practice in teacher training
Some states have created wage bonus policies to increase the attractiveness of the teaching profession, e.g. in Amazonas, Ceará, Rio de Janeiro, Minas Gerais, São Paulo and Pernambuco (Box 6) (Furtado and Soares, 2018[44]). These policies grant a wage bonus to teachers and sometimes to school administration officers. The bonus may relate to positive performance reviews, to the achievement of a certain grade in standardised exams or to students performance at the schools where they work. Research so far has found positive effects of wage bonus policies. In the states of São Paulo and Pernambuco, these policies had positive impacts on students’ performance, as measured by the Basic Education Evaluation System (Saeb) scale and by Prova Brasil results (Furtado and Soares, 2018[44]).
In 2009, Pernambuco launched an innovative pay for performance system that rewards school personnel for achieving their annual school improvement targets. All school personnel – not only teachers – of schools that achieve at least 50% of their targets receive a proportional bonus, up to a cap of 100%. Schools achieving less than 50% of their targets receive nothing, which functions as a strong incentive. In São Paulo, by comparison, every school receives some bonus.
In the first year of the program, 52% of schools achieved their targets, and the awards averaged 1.8 months of salary. In the second year, 79% of schools received the bonus, and the average award was 1.4 months of salary. This is a large incentive compared with other programmes internationally.
Research shows that the wage bonus programme is positively associated with higher results in the state evaluation system (Saepe) and in the Prova Brasil. However, the initial improvement in results induced by the programme did not remain constant over time. Once a given performance standard is reached, it is challenging for schools to sustain it. To address that, the state needs to design mechanisms to motivate school personnel and to assist schools that have consistently failed in reaching their targets.
Source: (Bruns, Evans and Luque, 2011[28]; Furtado and Soares, 2018[44]).
Dimension 5: Performance and evaluation in the education sector
The national system of monitoring and evaluation is essential to produce sound assessments and guide the formulation of policies towards equitable and quality education (Castro, 2002[55]). Indeed, as important as the existence of monitoring and evaluation systems, is the use of the results and the feedback loop to improve policies.
With the aim of generating context-specific information and to reach a greater level of detail, most states have developed their own evaluation systems. Municipalities, in their turn, tend to resort to the data produced by the federal and state governments. They do not develop their own evaluation systems not only because it would be costly, but mainly because the available ones already provide information disaggregated at municipal and school levels and sometimes even per student. Furthermore, municipalities may prefer to allocate resources in qualitative assessments of their education system, through questionnaires, interviews and reports. This type of assessment is usually undertaken in the diagnostic phase of their planning processes, be it the local education plan or the Co-ordinated Action Plan (PAR) discussed above.
Performance and evaluation assessments at the national level
The National Institute for Educational Studies and Research (INEP) is a federal agency specialised in education statistics and assessment. INEP is responsible for the School Census and other major educational assessments at the national level. It also manages the systems that compile and provide data on educational expenditures, namely the Educational Public Budget Information System (SIOPE) and the Finances of Brazil system (FINBRA) (Hirata and Oliveira, 2017[57]).
The School Census of basic education is carried out yearly and collects data to subsidise the planning and implementation of education policies at all levels of government. Public and private schools, together with state Secretaries of Education, assist INEP in conducting the Census. In return, each school receives a set of indicators, comprising the percentage of teachers with higher education, the age-grade gap, the average number of students per class, daily class hours, and promotion, repetition and drop-out rates (Castro, 2002[55]). In all, the information of the School Census enables education authorities to design and implement policies based on the main needs and demands of schools, as well as to monitor and evaluate the results of these policies.
The INEP is also responsible for the National Basic Education Assessment System (SAEB), which comprises several assessments, including the National Secondary Education Examination (ENEM), the Prova Brasil and the National Assessment of Literacy (ANA). In 2018, the Ministry of Education (MEC) unified the SAEB, the Prova Brasil and the ANA exams under the common appellation SAEB, but the content of each one remains the same as before.
SAEB assesses levels of student achievement and gathers information through questionnaires filled by students, teachers and school principals. This information includes socioeconomic data, students’ study habits, teacher training and practice and school administration (Castro, 2002[55]). Conducted every two years since 1995, it became a compulsory school-based system in 2005 (Hirata and Oliveira, 2017[57]).
In addition to test scores, the Basic Education Development Index (IDEB) was created in 2007. The IDEB is calculated from students’ performance measured by Prova Brasil, alongside with data on school passing and dropout rates. The index ranges from 0 to 10, allowing civil society and local governments to monitor, in simple terms, the performance of municipal and state schools.
ENEM was created to assess individuals – unlike SAEB, which was developed to compare different education systems. In 2009, the results of ENEM were adopted in the selection process of higher education institutions. ENEM is interdisciplinary, containing questions about all subjects taught at school, plus a written essay, whereas SAEB only evaluates learning on Mathematics and Portuguese.
Municipalities are called in to implement these assessments. Both public and private schools function as examination centres that apply the exams and then return the results to the federal government. This way, the INEP ensures the granularity of the system, while municipalities benefit from receiving the results disaggregated at school level.
State systems of evaluation and performance: the road ahead1
Since the early 1990s, states have invested in the creation of their own evaluation systems. The pioneer states were Ceará, Minas Gerais and Mato Grosso do Sul. These early systems had their own scale of evaluation, not aligned with the SAEB scale, which had been created in 1990. As of 2000, Paraná, Rio de Janeiro and Espírito Santo had created their systems, too. Progressively, states continued investing and adapting their systems, with technical assistance provided by the INEP. As of 2016, 18 out of the 26 states had created their own evaluation systems, following the SAEB scale.
These systems assess the performance of students in upper primary education and in lower and upper secondary education, in the disciplines of mathematics and Portuguese. Some of them assess the results in Sciences and Humanities as well. In addition, 11 of them evaluate literacy levels of students at the age of learning. These evaluations generate transversal results at the student, classroom, school and municipal levels.
There are great advantages in the development of local assessment programmes. They allow data breakdown at school level, collection of information of special interest and overall a deeper understanding of their specific context (Castro, 2002[55]). Furthermore, this process has fostered a multi-stakeholder debate about the right to education in states. State administrators, civil society and academia could better assess the factors that promote or hinder student performance. They were more consistently engaged in the design of innovative curricula, based on the deficiencies identified in the evaluations.
However, some challenges remain. The involvement of teachers in the analysis of results has been reported as weak. As those working directly in the classroom, teachers would have much to contribute to the analysis as well as learn from it. State schools in Ceará, for instance, use the results of mid-year standardised testing to adjust course contents, with the goal of increasing student performance levels before the end of the school year (Codes, A, 2018[38]) Even though a myriad of materials has been produced in order to facilitate the interpretation of results, these materials are not well-known by all the stakeholders involved. Lastly, and in relation to that, communication channels between those stakeholders could be strengthened.
Dimension 6: Territorial disparities in the education sector
Territorial disparities in the education sector are the most visible through the unequal availability and distribution of resources between municipalities, across and within states. To address these challenges, Brazil needs not so much to invest more in education, for the level of spending in relation to the GDP already lies above the OECD average, but to spend resources more efficiently (Oliveira et al., 2015[40]; Moreira, 2017[43]). This is why a sound multi-level governance framework that incorporates the territorial dimension in the elaboration and implementation of education policies is necessary. Acknowledging regional disparities is an important step towards better policy design and more efficient resource allocation. To overcome these disparities, the country has set up several equalisation systems.
The first equalisation system in Brazil was the Fund for the Development of Primary Education and Appreciation of Teachers (FUNDEF). This fund enabled a federally mandated system of redistribution within states, with a federally managed top fund (Bruns, Evans and Luque, 2011[28]). The FUNDEF regime also established that 60% of resources must be spent on teacher salaries, and the other 40% on operation costs. The impacts of this measure in raising teacher salaries were positive: in the northeast and north regions, a 70% increase was estimated (Bruns, Evans and Luque, 2011[28]).
In 2007, the federal government replaced the FUNDEF, which had a sunset clause, for the Fund for the Development of Primary Education and Appreciation of Teachers (FUNDEB). This enabled not only to give continuity to this policy, but also to expand it to the whole system of basic education, whereas before it concerned only primary schools. FUNDEB is ruled by Law nº 11.494/2007 and Decree nº 6.253/2007. States contribute to the fund with tax and transfer revenues, which are then shared across municipalities under an equalisation scheme, based on enrolment numbers (Santos et al., 2017[58]). This way, many municipalities that before fell under the minimum level of spending could reach it.
Therefore, because differences in contribution among states are complemented by federal funds, FUNDEB plays a very important role in reducing regional inequalities. Moreover, it addresses intra-state inequalities, due to the mechanism of redistribution across municipalities of the same state. It is estimated that the difference between the lowest and the highest student spending per year would be 350% bigger had the FUNDEB not been implemented (Cruz et al., 2019[59]).
There are, however, some elements showing that the FUNDEB’ design hinders further redistributive effects (Diniz, 2012[33]; Cruz et al., 2019[59]). For instance, based on the design of the fund, smaller municipalities, which have proportionally fewer pupils, end up financing, in part, the so-called winning municipalities (Diniz, 2012[33]). As a general rule, small municipalities are more significantly dependent on transfers from the federal government. Nonetheless, they still have to contribute to the fund, with the purpose of providing horizontal inter-municipal equalisation. Hence, when receiving the resources redistributed by FUNDEB, they end up being penalised (Diniz, 2012[33]).
To address that challenge without changing the underlying logic of resources being proportional to enrolment numbers, research findings suggest attributing more weight to poor students in the equalisation formula (Cruz et al., 2019[59]). That is, subnational governments would receive more funds not only proportionally to the size of their student population but also to the number of pupils with low socioeconomic status. This mechanism could compensate for social inequalities, at least partially.
Secondly, the parameter of student spending per year precludes considerations about the efficiency of spending. Several proposals have been made, including amendments to the Constitution, about establishing the quality of education as a parameter to redistribute FUNDEB funds (Cruz et al., 2019[59]). The adoption of an index of Cost Pupil Quality (CAQ) is foreseen in the Strategies 20.6 and 20.7 of the Law nº 13.005/2014.
The Cost Pupil Quality (CAQ) would operate as a reference of how much subnational governments would have to spend on each level of education per student in order to deliver quality education (Cruz et al., 2019[59]). Today, the reference for redistribution is how much subnational governments spend on their budget, which of course depends on their tax regime and political willingness, while not telling much about the efficiency of resource allocation. The CAQ index seeks to compensate for that, by adding the dimension of quality, which has become an important directive for Brazil’s education policy, at least since the 1996 Law of Directives and Bases of Education (LDB).
Brazil has a complex multi-level institutional and financial architecture. The quality of multi-level governance can have great impact on the delivery of decentralised policies in Brazil. When auditing these policies an assessment of the multi-level governance dimension is particularly crucial.
To direct the assessment of the MLG dimension in an audit TCs could, as a pre-audit study, develop an assessment framework, by following these steps:
describing the multi-level governance system for a policy area and identify key dimensions that could be assessed during the audit
defining key maturity indicators for each of these dimensions, stating unit of analysis; description and rationale of the indicator; information and sources needed for assessment; and diagnostic question for evaluating the level of maturity
specifying the different levels of maturity for each indicator, distinguishing between basic, partially achieved and good
applying the MLG assessment framework to plan and design the audit.
Following these steps, and based on the MLG generic assessment framework developed by the OECD, TCs may develop MLG assessment frameworks for any policy sector(s) that will be audited. These framework(s) may contain the six dimensions of the generic framework – but should not be restricted to them:
1. assignment of responsibilities
2. funding of subnational responsibilities
3. capacities of subnational governments and capacity building
4. co-ordination among levels of government
5. performance monitoring and transparency
6. fiscal equalisation systems and regional policies to reduce territorial disparities.
These dimensions should not be considered an exhaustive list of the multi-level governance challenges in Brazil; they can be revised and refined depending on the needs and evolution of the country and the TCs while following the four steps detailed above. Steps 1-3 of the described model do not attempt at providing an exhaustive assessment of the MLG system for a selected policy sector, but a detailed description that could serve as a basis to conduct this assessment during the audit process.
The framework below presents all the maturity indicators developed by the OECD for the generic MLG assessment framework. The different maturity levels were defined based on the 10 guidelines for making decentralisation work (OECD, 2019[3]) and the OECD Recommendation on Effective Public Investment across levels of government (OECD, 2019[2]).
Dimension 1: Assignment of responsibilities
Criteria: Clear definition of responsibilities across levels of government
Unit of Analysis: All levels of government
Criteria description (rationale): A clear and transparent division of powers implies that the responsibilities of various levels of government must be codified in significant detail in legal and regulatory frameworks, and/or intergovernmental agreements, among others. The more a responsibility area is shared across different government levels, the greater clarity is needed to reduce duplication and overlaps, particularly in federal contexts.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: To what extent do the law/regulations clearly specify the responsibilities of the different levels of government?
Maturity levels
Criteria: Clear and coherent allocation of functions across levels of government
Unit of Analysis: All levels of government
Criteria description (rationale): In addition to the clear definition of responsibilities, it is important to clarify each sub-function, i.e. the role of various levels of government in policy, legislation, standards, oversight, financing, provision/administration, production, distribution, performance monitoring, evaluation, citizen complaints, feed-back and redress mechanisms.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: To what extent do the law/regulations clearly specify the functions of the different levels of government in relevant policy area?
Maturity levels
Criteria: Municipal spending autonomy
Unit of Analysis: Subnational governments
Criteria description (rationale): Spending autonomy is determined as the degree of autonomy and decision-making authority of subnational governments in their fields of responsibility. Spending autonomy is low when local governments act as agencies funded and regulated by the central government rather than as independent policymakers. It can also be limited if subnational service provision is strongly steered by normative regulation set by a higher level of government without consulting lower levels of government.
Source of information: Field work (interviews, focus groups), law and regulations, previous audits.
Diagnostic question: To what extent subnational governments have autonomy and decision-making authority in their areas of responsibility?
Maturity levels
Criteria: Accountability of subnational governments
Unit of Analysis: Subnational governments
Criteria description (rationale): In decentralised systems, elected local authorities are often accountable to residents who finance and consume the services. Mechanisms to assess accountability of governments include local and national elections, but also other channels for citizens to express their voices, such as participation in surveys, town meetings, local referenda and direct involvement in service delivery. Large variations exist across countries in terms of responsibilities and local decision-making powers, resulting in varying degrees of accountability, and varying ranges of central government control.
Source of information: Law, regulations, surveys, minutes of town meetings, perception surveys, previous audits.
Diagnostic question: To what extent are local governments accountable for the delivery of services under their area of responsibilities?
Maturity levels
Criteria: Public service provision coverage
Unit of Analysis: Municipal government
Criteria description (rationale): According to the principle of subsidiarity, taxing, spending and regulatory authority for any service should be vested in the lowest order of government, unless a convincing case can be made for higher order assignment (lack of capacity or resources). To assess this criteria, the allocation of competences should ensure that minimum standards for service coverage are met by each local government throughout the country.
Source of information: Field work (interviews, focus groups), public service assessments, previous audits.
Diagnostic question: To what extent is there a homogenous coverage of public services over the whole territory at the local level?
Maturity levels
Criteria: Citizens’ engagement
Unit of Analysis: All levels of government
Criteria description (rationale): Subnational public governance systems can facilitate the participation and engagement of private citizens and other stakeholders in deliberations on public policy choices and the delivery of local public services. The modalities for citizens’ engagement vary depending on local contexts, including tools for citizens to express their opinions on local services and problems.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits
Diagnostic question: Are citizens engaged in the decision-making process at all levels of government?
Maturity levels
Dimension 2: Funding of subnational governments
Criteria: Funding of core responsibilities
Unit of Analysis: Subnational governments
Criteria description (rationale): For a decentralisation model to be effective, responsibilities for spending (expenditure needs) must be sufficiently funded. This means that local governments’ revenue means (including own revenues, shared taxes and transfers) must be consistent with expenditure needs, for each level of government to discharge its public service responsibilities consistent with its mandate. This consistency between revenue-generating means with expenditure needs is a factor of political accountability and responsiveness to local preferences.
Source of information: Law, subnational financial accounts and field work (interviews, focus groups), previous audits.
Diagnostic question: Do subnational governments own revenues finance a large share of their expenditure?
Maturity levels
Criteria: Tax autonomy
Unit of Analysis: Subnational governments
Criteria description (rationale): Tax autonomy depends on many factors, including the ability of local government to set or modify tax rates and bases (in addition to revenue from transfers, tariffs, user charges or fees or from local assets). There is no general rule for an optimal degree of tax autonomy, but local authorities should generally rely on their own revenues for financing their services at the margin. A particular challenge for the central government consists in setting up the vertical distribution of tax revenues among levels of government and to determine which taxes to assign to subnational governments, under what criteria, and with which degree of discretionary power over tax bases and rates.
Source of information: Law, subnational financial strategies, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the municipality has autonomy over its tax base?
Maturity levels
Criteria: Intergovernmental transfers
Unit of Analysis: All levels of government
Criteria description (rationale): The extent of discretion in intergovernmental transfers may vary, from earmarked and conditional transfers to general-purpose grants based on a formula. In general, municipalities with limited fiscal capacity are more reliant on intergovernmental transfers, which must be stable, predictable, and based on transparent and well-defined criteria, adapted to regional and local specificities.
Source of information: Law, subnational financial accounts, field work (interviews, focus groups).
Diagnostic question: Are intergovernmental transfers to the municipality designed and implemented in a stable and regular basis?
Maturity levels
Criteria: Municipal financial management
Unit of Analysis: All levels of government
Criteria description (rationale): Financial management capacity refers to the ability of subnational governments to ensure the effective use of internal and external resources with integrity. This includes budget preparation, cash management, transparent procurement processes to mitigate corruption, how to use internal controls and internal and external audits to ensure efficiency and integrity. Proper costing and budgeting can also serve to prioritise and execute investment programmes effectively.
Source of information: Fiscal rules and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the municipality apply best practices in budget management and budget reporting?
Maturity levels
Criteria: Use of Innovative financing mechanisms
Unit of Analysis: Subnational governments
Criteria description (rationale): In cases where public sources of funding are insufficient to cover expenditure and investment needs at the state government and local government levels, diversifying revenue sources by resorting to external sources of financing and innovative financing mechanisms can help cover this gap. Local and state governments can mobilise innovative sources of financing through partnering with the private sector and institutional investors, issuing bonds, Public-Private Partnerships, joint borrowing in capital markets or other instruments such as green bonds and social bonds.
Source of information: Subnational financial accounts, field work (interviews, focus groups), previous audits.
Diagnostic question: Does the local government access innovative sources of financing for financing expenditure and investment needs?
Maturity levels
Criteria: Municipal debt and borrowing
Unit of Analysis: Subnational governments
Criteria description (rationale): The ability of governments to decide on how and when to use debt, how to assess debt affordability, what debt to use, how to issue and how to manage debt is crucial. This is particularly the case for subnational government, considering that if the level of subnational debt is too high, this may have repercussions on the sustainability of public finances and the capacity of local governments to finance future needs. Municipal borrowing capacity can be determined by the extent to which a local government can borrow, as well as budget constraints and fiscal rules applicable to local governments
Source of information: Law, national and subnational financial accounts, field work (interviews, focus groups).
Diagnostic question: To what extent can the municipality borrow in a sustainable way?
Maturity levels
Dimension 3: Capacities and capacity building
Criteria: Subnational strategic capacity
Unit of Analysis: Subnational governments (states and/or municipalities)
Criteria description (rationale): Strategic capacity refers to the ability to set strategic goals for social, political and economic outcomes and having the administrative and institutional capacity to realise those goals within a stated period. Strategic capacity is especially critical for local and regional development strategies that require substantial citizen input and co-ordination across and beyond governments. The lack of sufficient technical or strategic capacities is one of the bigger challenges in the field of decentralisation, as building capacities takes time and needs a long-term commitment from central and subnational governments.
Source of information: Municipal/state strategic documents, field work (interviews, focus groups), previous audits.
Diagnostic question: Does the local government have the strategic capacity to set realistic objectives and goals, and develop strategic plans for local economic development?
Maturity levels
Criteria: Staffing needs
Unit of Analysis: Subnational government
Criteria description (rationale): The good allocation of staff is essential to ensure the delivery of results at the level of the government administration. This depends to a large extent on the staff selection process, and the types of evaluations that are conducted to assess staff performance and contributions. It is recommended to ensure open, competitive hiring and merit-based promotion to meet these goals.
Source of information: Field work (interviews, focus groups).
Diagnostic question: Has the local government put in place a staff hiring process able to meet its capacity needs?
Maturity levels
Criteria: Capacity-building for subnational policies
Unit of Analysis: All levels of government
Criteria description (rationale): Local governments may face growing pressure to increase their size and capacity when they are transferred to an increasing number of tasks, in order to cope with the additional responsibilities. To face this increase, it is necessary to strengthen government capacities at the subnational level, with capacity building policies tailored to the various needs of subnational governments. Building and strengthening subnational capacities is a long-term commitment, which requires sustained resources and political commitment from both subnational and central/federal government levels, including the right framework conditions for decentralisation to be in place.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the local government benefit from capacity-building programmes adequately funded and planned over the long-term?
Maturity levels
Criteria: Technical guidance documents
Unit of Analysis: All levels of government
Criteria description (rationale): An effective way to support capacity-building at the subnational level is to distribute formal/standardised guidance documents in areas such as planning, project appraisal, procurement, or monitoring and evaluation.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Are technical guidance documents available to local governments to guide them in subnational policy areas?
Maturity levels
Criteria: Digital tools
Unit of Analysis: Subnational governments
Criteria description (rationale): Digital tools can support the work of subnational governments in a number of ways. Information and communication technology (ICT) and most recent technological changes (blockchain, robotics) have multiplied the opportunities for local governments to improve the ways they communicate and involve citizens (e.g. e-democracy and ICT-based participation). Digital tools also help governments provide local public services (e-government), manage public resources in a more efficient manner (e.g. for tax collection), improve staff capacity and management and adopt new public management models. Digital tools can finally improve the relationships between the central and subnational governments, facilitating the shift towards more decentralised governance practices.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the local government use digital tools in administrative tasks and service provision?
Maturity levels
Criteria: Subnational capacities for public investment
Unit of Analysis: Subnational governments
Criteria description (rationale): The lack of capacity to design and implement the right investment-mix is often a main bottleneck at the subnational level. Defining, structuring, implementing, operating and monitoring public investment requires a very diverse set of capacities, in particular for infrastructure investments, which are unevenly distributed among the territories. Specific skills and expertise is needed from within the staff to perform these functions in an effective way.
Source of information: Municipal investment strategies and plans, field work (interviews, focus groups), previous audits.
Diagnostic question: Does the local government have sufficient capacity to plan and implement public investment?
Maturity levels
Criteria: Horizontal co-ordination and knowledge-sharing for capacity-building
Unit of Analysis: All levels of government
Criteria description (rationale): Local governments and municipalities are not all and not always equipped with the same level of capacity to ensure successful service delivery and policy implementation, as there can be important differences between subnational governments in financial capacity and administrative skills (in terms of staff, expertise, scale). This can lead to increase disparities among local governments themselves. To reduce these disparities, horizontal co-ordination can be carried out by promoting inter-municipal, interregional co-operation, or metropolitan governance for instance. Through rural-urban partnerships in particular, rural local governments can benefit from the resources and expertise from their urban counterparts, and both parties can strengthen their capacities. The legal system at the national level should allow such tools.
Source of information: Law, regulations, and field work (interviews, focus groups).
Diagnostic question: Does the local government participate in peer-to-peer sharing platforms or other institutions of horizontal co-ordination, with the objective of strengthening its capacities?
Maturity levels
Dimension 4: Co-ordination among levels of government
Criteria: Consistent legal and regulatory frameworks
Unit of Analysis: All levels of government
Criteria description (rationale): In a decentralised context, the central government has to provide an overarching framework and guidelines for sectoral policies and strategies, in order to ensure that policies that are designed and implemented at all levels are driven by a common goal and do not follow contradictory objectives. Consistent legal and regulatory frameworks should be aligned at all levels to ensure that objectives are met.
Source of information: Law, regulations.
Diagnostic question: Are frameworks at the national, regional and municipal level aligned in each policy sectors?
Maturity levels
Criteria: Vertical co-ordination for policy planning
Unit of Analysis: All levels of government
Criteria description (rationale): In a decentralised context and in federal countries in particular, unilateral decisions without consultation may undermine trust. It is therefore important to find the right balance between top-down and bottom-up approaches. Intergovernmental platforms for vertical co-ordination have the potential to help clarify, co-ordinate, and develop reform options, joint provision or partnership arrangements for tax, expenditure, revenue sharing and transfers, public services delivery and regulatory policies. They make take the form of dialogue platforms, fiscal councils, standing commissions and intergovernmental consultation boards.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Are there tools and platforms for vertical co-ordination across levels of government?
Maturity levels
Criteria: Co-ordination arrangements for financing
Unit of Analysis: All levels of government
Criteria description (rationale): Formal co-financing arrangements instruments (e.g. contract, associations) are helpful to build trust between levels of governments. In some cases, there can be single or multi-purpose co-operative agreements/contracts (e.g. shared services arrangements in healthcare or education). These arrangements also provide a long-term perspective for the financing and investment strategies of local and regional governments through long-term service outsourcing.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Is the subnational government engaged in co-financing arrangements?
Maturity levels
Criteria: Multi-year programming
Unit of Analysis: Subnational governments
Criteria description (rationale): A multi-year approach to policy-making, and in particular investment policies, can support local government’s capacity to prioritise its expenditure and investment programmes, and ensure that they are aligned with priorities set at the regional and national levels of government. Multi-year programming is done by connecting planning and budgeting frameworks, and can also help provide visibility regarding resource availability and predictability, particularly for financing long-term projects, which may need to survive changes of government.
Source of information: Municipal and state governments budgets, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the subnational government develop multi-year investment programming and medium-term budget forecasts?
Maturity levels
Criteria: Participatory/Citizens budgets
Unit of analysis: Municipalities
Criteria description: Participatory Budgeting is a process whereby citizens directly participate in the allocation of a defined part of a government’s budget. They are initiatives are typically initiated by local government authorities, frequently in response to demands from community groups, CSOs and/or citizens for a greater say in deciding how and where public resources should be spent. Participatory budgeting creates opportunities for educating and empowering citizens and for strengthening citizen-government relations. It also helps to promote government transparency and accountability, and the responsiveness and effectiveness of government programmes and services.
Source of information: field work (interviews, focus groups), websites, previous audits.
Diagnostic question: Does the municipality conduct participatory budget initiatives to respond to citizen’s demands and promote transparency and accountability?
Maturity levels
Criteria: Resolution of conflict between municipal, state and central administrations
Unit of Analysis: All levels of government
Criteria description (rationale): When powers and responsibilities are shared and concurrent, a legal framework must establish which level of government has legislative supremacy in the event of a conflict. If multiple levels of government have exclusive jurisdiction in sub-areas of a function (for example, education, health or environment impact assessment), then inter-governmental agreements must specify the precise processes to reach an agreement and to resolve conflicts in allowing projects to proceed. In the absence of such clarity, critically important projects in specific policy areas may be unduly delayed, or even abandoned.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the legal framework provide adapted mechanisms to reach an agreement in case of conflict between various levels of government?
Maturity levels
Criteria: Horizontal co-ordination for service provision
Unit of Analysis: All levels of government
Criteria description (rationale): Horizontal co-ordination is essential to increase efficiency through economies of scale and to enhance synergies among policies of neighbouring (or otherwise linked) subnational governments. Horizontal co-ordination mechanisms can take a variety of forms (e.g. inter-municipal co-operation, mergers), depending on the characteristics of the locality and policy objectives. In addition, horizontal co-ordination can be directly encouraged and rewarded by higher levels of governments through incentives, grants, etc.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the municipality develop mechanisms for horizontal co-ordination for service provision?
Maturity levels
Criteria: Cross-jurisdictional partnerships involving investment
Unit of Analysis: Subnational governments
Criteria description (rationale): To ensure that public investments are made at the relevant scale, horizontal co-ordination is crucial. It is important for physical infrastructure provision where the efficient scale often exceeds the administrative boundaries of individual local governments, and for investments in human capital development and innovation where administrative and functional boundaries may not coincide. For these reasons, governments should provide incentives and/or seek opportunities for horizontal co-ordination across regions and/or local governments to match public investment with the relevant geographical scale (e.g. through contracts, specific public investment partnerships, joint authorities, or regional or municipal mergers).
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the local government seek to develop cross-jurisdictional partnerships for co-ordinating investment with other local governments?
Maturity levels
Dimension 5: Performance measurement and transparency
Criteria: Ex ante analysis to inform decision-making at different stages of the policy process
Unit of Analysis: Subnational government
Criteria description (rationale): Ex ante appraisal helps identify the long-term impact and risk of public spending and investment upfront in the policy cycle. It can support identifying the social, environmental and economic impact of public spending, while also assessing which method can yield the best value for money. To be rigorous and maximise its effects, ex ante appraisal must be conducted by staff with the necessary skills, be subject to an independent review, and follow high-quality technical guidelines.
Source of information: Law, guidance documents for ex ante analysis, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the subnational government conduct ex ante analysis to prioritise and assess the impacts of public spending?
Maturity levels
Criteria: Ex post evaluations to improve performance all along the policy cycle
Unit of Analysis: All levels of government
Criteria description (rationale): The prioritisation of projects at the municipal level can be improved through ex post evaluations and by developing forward-looking scenarios. Ex post evaluation addresses the goals of policies, seeking to determine if intended outcomes were achieved, and the role played by public expenditure and investment activities. The central government can support local governments through this process by requiring and/or co-financing ex post evaluations at the subnational level, and by setting up an independent institution at the national level to carry out these evaluations.
Source of information: Law, guidance documents for ex post evaluation, and field work (interviews, focus groups), previous audits.
Diagnostic question: Are ex post evaluations conducted by the subnational government, based on clear ex post evaluation standards?
Maturity levels
Criteria: Outcome-based performance systems for social policies
Unit of Analysis: All levels of government
Criteria description (rationale): Outcomes-oriented public policy strategies focus on the outcome and performance of expenditure and investments. To assess this performance, the monitoring system must use criteria for service outcomes. Evaluation and monitoring criteria need to be defined in the early stages of the policy design, in order to allocate the resources needed and produce the appropriate data for this purpose. In addition, governments at all levels need the adequate capacity to monitor these criteria and adjust them if needed, based on the local contexts.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the performance monitoring system of the local government uses criteria for service outcomes?
Maturity levels
Criteria: Guidance documents for rigorous monitoring and evaluation
Unit of Analysis: All levels of government
Criteria description (rationale): Monitoring and evaluation are important mechanisms for accountability and learning: for transferring knowledge among parties and for improving performance by integrating feedback in each policy area. Clear guidance documents should be provided to local governments to set up ex post evaluation standards.
Source of information: Law, guidance documents and field work (interviews, focus groups) , previous audits.
Diagnostic question: Do local governments have the necessary guidelines and tools for conducting effective monitoring and evaluations?
Maturity levels
Criteria: ICT tools and new technologies for performance evaluation
Unit of Analysis: All levels of government
Criteria description (rationale): Local governments can be encouraged to rely on an extensive set of mechanisms for performance evaluation, including ICT and e-government tools (e.g. ICT platforms). ICT and e-government tools can be used for tracking enquiries/transactions, measuring service delivery response times and surveying customers. ICT tools can also help government to better understand who the service users are and to learn about their needs.
Source of information: Municipal practices and field work (interviews, focus groups)
Diagnostic question: Do local governments use ICT and e-government tools to measure, monitor and disseminate their own performance?
Maturity levels
Criteria: Benchmarking analysis for performance measurement
Unit of Analysis: All levels of government
Criteria description (rationale): Following the growing interest in international benchmarking of performance across the OECD, all levels of governments can produce benchmarking analysis in selected policy areas (e.g. health, education), at the inter-local, regional and/or international level, and use the information available for performance measurement. To this end, information among local governments must be accessible openly, and updated regularly.
Source of information: Law, and field work (interviews, focus groups), previous audits.
Diagnostic question: Does the local government perform benchmarking analysis to assess its performance measurement?
Maturity levels
Criteria: Transparency and reporting
Unit of Analysis: All levels of government
Criteria description (rationale): All levels of governments should be committed to consistently improve transparency, enhance data collection and strengthen their performance monitoring systems. Transparent and freely available information enables performance measurements and benchmarking between governments and their agencies and entities. To this end, it is important to ensure that evaluations are disseminated and reported to the citizens, higher-order governments, private sector and other interested parties.
Source of information: Law, regulations, and field work (interviews, focus groups), previous audits.
Diagnostic question: Are evaluations disseminated openly to public and private stakeholders?
Maturity levels
Criteria: Availability and transparency of administrative procedures
Unit of analysis: State/municipalities
Criteria description: A majority of procurement is undertaken at the subnational level and at the same time it is one of the government activities most vulnerable to waste, fraud and corruption. Transparency throughout the procurement cycle, professionalisation of the procurement function, and clear accountability and control mechanisms are all required.
Source of information: Field work (interviews, focus groups), e-tools, websites, previous audits.
Diagnostic question: Are the administrative procedures for public procurement transparent and clear for citizens and business?
Maturity levels
Dimension 6: Territorial inequalities
Criteria: Fiscal equalisation focusing on fiscal needs
Unit of Analysis: All levels of government
Criteria description (rationale): Fiscal need equalisation can be achieved through a demand for services approach that allocates funds by service population, e.g. school-age population for school finance. Alternately, fiscal need equalisation can be achieved through output-based sectoral grants that also enhance results-based accountability. It is recommended to establish separate formula allocations for each type of municipality/local government, acknowledging population size, the area served and the urban/rural nature of services in making grants to local governments.
Source of information: Legal texts, subnational government financial accounts, interviews, previous audits.
Diagnostic question: Is the fiscal capacity of the local government equalised based on its fiscal needs?
Maturity levels
Criteria: Simple and clear fiscal equalisation system
Unit of analysis: All levels of government
Criteria Description: In the design of fiscal equalisation transfers, rough justice is better than precise or full justice if it achieves wider acceptability and sustainability. It is important to focus on a single objective (equalisation for equalisation transfers) in a grant programme and make the design consistent with that objective. Setting multiple objectives in a single grant programme runs the risk of failing to achieve any of them.
Source of information: Legal texts, subnational government financial accounts, interviews, previous audits.
Diagnostic question: Has the fiscal equalisation system in place a single and clear objective?
Maturity levels
Criteria: Regional development policies to reduce inequalities
Unit of Analysis: All levels of government
Criteria description (rationale): In order to offset the potentially negative incentives of equalisation systems, these systems need to be accompanied by pro-active regional development policies, implemented at the national, regional and local levels.
Source of information: National, regional and local development plans, interviews, previous audits
Diagnostic question: Are there active regional development policies and incentives in place to reduce disparities at the local government level, especially between rural and urban areas?
Maturity levels
Dimension 1: Responsibilities clearly assigned and accountable
Criteria: Clear definition of responsibilities across levels of government in the sector of education
Unit of Analysis: State and municipal levels of government
Source of information: 1988 Constitution (arts. 21 to 24 and arts. 205 to 214), Law of Bases and Directives of Education (LDB) (Law nº 9.394/1996), Law of the National Education Plan (PNE) (Law nº 13.005/2004) and All for Education Commitment Plan and Co-ordinated Action Plan (PAR) legislation (Decree nº 6.094/2007).
Diagnostic question: Do the law/regulations clearly specify the responsibilities of the different levels of government in the sector of education?
Maturity levels
Criteria: Municipal spending autonomy
Unit of Analysis: Municipal level of government
Source of information:
1988 Constitution (arts. 205 to 214), Law of Bases and Directives of Education (LDB) (Law nº 9.394/1996, especially articles 68 to 77), FUNDEB Law (Law nº 11.494/2017), Wage Floor Law (Law nº 11.738/2008).
National Tax Code (CTN) (Law nº 5.172/1966, especially arts. 32 to 34, 77 to 79, 83 to 85 and 91).
Each Municipality has its Municipal Organic Law, Tax Code, Administrative Organisation Law and Law of Local Education Plan.
Diagnostic question: Do local governments have full autonomy and decision-making authority to deliver primary and lower secondary education?
Maturity levels
Criteria: Coverage of public services related to education
Unit of Analysis: Municipal and state level of government
Source of information:
Local Education Plan and other relevant local plans related to public service provision (Transportation plan).
Documents at state and federal levels regarding transportation arrangements.
Diagnostic question: Is the provision of education services homogenous and well-adapted to the whole territory of the subnational government’s jurisdiction?
Maturity levels
Dimension 2: Responsibilities sufficiently funded
Criteria: Intergovernmental transfers earmarked for education
Unit of Analysis: All levels of government
Source of information:
1988 Constitution (arts. 205 to 214) and Law of Bases and Directives of Education (LDB) (Law nº 9.394/1996, especially articles 68 to 77).
FUNDEB Law (Law nº 11.494/2017) and Educational Allowance Law (Law nº 9.424/1996 and Law nº 9.766/1998).
National Programme to Support School Transportation (PNATE) (Law nº 10.880/2004), National Programme of School Meals (PNAE) (Law nº 11.947/2009), Direct Money at School Programme (PDDE) (Law nº 11.947/2009) and National Programme of Textbooks (PNLD) (Decree nº 91.542/1985).
Field research with interviews and questionnaires to assess the balance between funding and assigned responsibilities.
Diagnostic question: Are intergovernmental transfers to municipalities for the education sector designed and implemented on a stable and regular basis?
Maturity levels
Criteria: Subnational financial management and resource mobilisation for funding the education sector
Unit of Analysis: State and municipal levels of government
Source of information: Municipal and state government financial accounts, Fiscal rules and field work (interviews, focus groups).
Diagnostic question: Does the subnational government apply best practices to optimise its budget management in the education sector?
Maturity levels
Dimension 3: Co-ordination among and across levels of government
Criteria: Vertical co-ordination for policy planning through the Co-ordinated Actions Plans (Planos de Ações Articuladas – PAR)
Unit of Analysis: All levels of government
Source of information:
Law of the National Education Plan (PNE) (Law nº 13.005/2004) and All for Education Commitment Plan and Co-ordinated Action Plan (PAR) legislation (Decree nº 6.094/2007).
National Common Curricular Basis.
National Education Forum (MEC Ordinance nº 1.407/2010).
Intergovernmental Commission on Financing for Quality Basic Education (arts. 12 and 13 of FUNDEB Law nº 11.494/2007).
Social Monitoring and Control Councils (CACS Fundeb) (arts. 24 to 30 of FUNDEB Law nº 11.494/2007).
Ministry of Education information systems, e.g. the SIMEC.
Field research with interviews and questionnaires to assess the levels of engagement in the participatory planning process.
Diagnostic question: Is the subnational government resorting to Co-ordinated Action Plans (Planos de Ações Articuladas – PAR) as a strategic participatory tool?
Maturity levels
Criteria: Co-ordination arrangements for financing education projects
Unit of Analysis: All levels of government
Source of information: Law, regulations, and field work (interviews, focus groups)
Diagnostic question: Is the local government engaged in co-financing arrangements with other levels of government?
Maturity levels
Criteria: Horizontal co-ordination for education provision
Unit of Analysis: Subnational governments
Source of information:
Inter-municipal consortium agreement, if any.
Educational Development Arrangement, if any (CNE Resolution nº 1/2012).
Forum of information sharing and peer learning, if any.
Joint entry exam procedures to hire teachers, if any.
Diagnostic question: Does the subnational government develop mechanisms for horizontal co-ordination for education provision?
Maturity levels
Dimension 4: Capacities and capacity building
Criteria: Municipal strategic capacity to guide education policies
Unit of Analysis: Municipal governments
Source of information:
Municipal Education Plan, Local Curricular Basis, Administrative Organisation Law.
Municipal Co-Ordinated Action Plan (Plano de Ações Articuladas – PAR).
Education Development Arrangement collaboration term, if any.
State programmes of technical assistance and training.
Diagnostic question: Does the local government have the strategic capacity to set realistic objectives, and develop strategic plans for the provision of education?
Maturity levels
Criteria: Digital tools
Unit of Analysis: Local governments
Source of information: Digital Governance Strategy (Decree nº 10.332/2020).
Diagnostic question: Does the local government use digital tools in administrative tasks, education provision and evaluation of educational results?
Maturity levels
Criteria: Capacity-building for subnational policies
Unit of Analysis: Municipal level of government
Source of information:
Municipal training programmes for teachers and professionals in the education sector.
Education Development Arrangement collaboration term, if any.
State programmes of technical assistance and training.
Federal programmes of technical assistance and training, e.g. the Support Programme to Municipal Education Officers (Programa de Apoio aos Dirigentes Municipais de Educação – Pradime).
PAR Term of Agreement between the municipality and the MEC.
Diagnostic question: Does the local government benefit from capacity building programs targeted to education policies, adequately funded, and planned over the long-term?
Maturity levels
Dimension 5: Performance and evaluation
Criteria: Outcome-based performance systems for education
Unit of Analysis: All levels of government
Source of information:
National evaluation system (art. 11 of Law nº 13.005/2014, of the PNE).
National Policy of Educational Assessment and Evaluation (Decree nº 9.432/2018).
Local education plans.
State and municipal information and evaluation systems.
Diagnostic question: Does the performance monitoring system of the local government uses indicators to measure education outcomes?
Maturity levels
Criteria: Guidance documents for rigorous monitoring and evaluation
Unit of Analysis: All levels of government
Criteria description (rationale): Monitoring and evaluation are important mechanisms for accountability and learning: for transferring knowledge among parties and for improving performance by integrating feedback in each policy area. Clear guidance documents should be provided to local governments to set up ex post evaluation standards.
Source of information:
National Policy of Educational Assessment and Evaluation (Decree nº 9.432/2018).
Guidelines for participation in the SAEB evaluation (INEP Ordinance nº 366/2019).
Diagnostic question: Do local governments have the necessary guidelines and tools for conducting effective monitoring and evaluations?
Maturity levels
Dimension 6: Addressing territorial inequalities
Criteria: Fiscal equalisation
Unit of Analysis: All levels of government
Source of information:
1988 Constitution (arts. 205 to 214), Law of Bases and Directives of Education (LDB) (Law nº 9.394/1996, especially articles 68 to 77) and Law of the National Education Plan (PNE) (Law nº 13.005/2004, especially art. 11).
FUNDEB Law (Law nº 11.494/2017) and Educational Allowance Law (Law nº 9.424/1996 and Law nº 9.766/1998).
Diagnostic question: Is the fiscal capacity of the subnational government in the education sector equalised based on its fiscal needs?
Maturity levels
References
[51] Abrucio, L. F (2017), Cooperação intermunicipal: experiências de arranjos de desenvolvimento da educação no Brasil. Curitiba: Positivo e Colabora Educação.
[52] Abrucio, L., C. Pereira and M. Seggato (2017), Regime de Colaboração no Ceará: Funcionamento, Causas do Sucesso e Alternativas de Disseminação do Modelo, Instituto Natura, http://www.institutonatura.org.br/wpcontent/uploads/2016/12/Relatorio-Ceara-AF-Web.pdf.
[9] Algemene Rekenkamer (2009), The system of supervision of financial markets, http://rekenkamer.presurf.nl/public/20200803151945/http:/www.rekenkamer.nl/.
[8] Algemene Rekenkamer (2000), Corporate bodies with a statutory task: reporting procedures and supervision, part 1, http://rekenkamer.presurf.nl/public/20200803151945/http:/www.rekenkamer.nl/.
[12] Allain-Dupré, D. (2018), “Assigning responsibilities across levels of government: Trends, challenges and guidelines for policy-makers”, OECD Working Papers on Fiscal Federalism, No. 24, OECD Publishing, Paris, https://dx.doi.org/10.1787/f0944eae-en.
[39] Assembleia legislativa do Estado do Sao Paulo (n.d.), Projeto de lei nº 855 /2019, https://www.al.sp.gov.br/propositura/?id=1000278839 (accessed on 1 June 2020).
[54] Barros, M. (2018), O elo institucional regional da educação básica: um estudo das diretorias regionais de ensino paulistas., https://bibliotecadigital.fgv.br/dspace/handle/10438/24071.
[28] Bruns, B., D. Evans and J. Luque (2011), Achieving world-class education in Brazil: The next agenda, World Bank.
[47] Camara dos Deputados (n.d.), Projetos de Lei e Outras Proposições, https://www25.senado.leg.br/web/atividade/materias/-/materia/139285 (accessed on 1 June 2020).
[46] Campanha Nacional pelo Direito à Educaçao (n.d.), Sistema Nacional de Educação, https://campanha.org.br/o-que-fazemos/sistema-nacional-de-educacao/ (accessed on 13 May 2020).
[27] Carnoy, M. et al. (2017), Intranational Comparative Education: What State Differences in Student Achievement Can Teach Us about Improving Education – the Case of Brazil, https://www.journals.uchicago.edu/doi/abs/10.1086/693981.
[31] Castro, M. H. G. (2010), Financiamento da educação pública no Brasil: evolução dos gastos, UNESCO, Brasilia.
[55] Castro, M. (2002), Evaluation: an educational reform strategy in Brazil, INEP, http://inep.gov.br/informacao-da-publicacao/-.
[48] CNE (2012), Parecer nº 11/2012, sobre o regime de colaboração entre os sistemas de ensino, http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=11616-pcp011-12-pdf&category_slug=setembro-2012-pdf&Itemid=30192.
[53] CNE (2012), Resolução nº 01, de 23 de janeiro de 2012, que dispõe sobre a implementação do regime de colaboração mediante Arranjo de Desenvolvimento da Educação (ADE), como instrumento de gestão pública para a melhoria da qualidade social da educação, http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=9816-rceb001-12&category_slug=janeiro-2012-pdf&Itemid=30192.
[38] Codes, A (2018), Lições de experiências exitosas para melhorar a educação em regiões com baixos índices de desenvolvimento, IPEA, http://repositorio.ipea.gov.br/handle/11058/9449.
[21] Council of Europe (2010), Toolkit Manual Inter-Municipal Co-operation.
[59] Cruz, T. et al. (2019), Novo Fundeb: Prós e Contras das Propostas em Debate, Inter-American Development Bank (IDB), https://doi.org/10.18235/0001853.
[33] Diniz, J. (2012), Eficiência das transferências intergovernamentais para a educação, Universidade de São Paulo, https://repositorio.usp.br/item/002292418.
[34] Dourado L. F. (2013), Sistema Nacional de Educação, federalismo e os obstáculos ao direito a educação básica., Educ. Soc., Campinas, v. 34, n. 124, p. 761-785, jul.-set. 2013.
[49] Fermandes, J. and J. Nogueira (2018), Plano de Ações Articuladas: Melhoria da educação e redução das desigualdades, Implementação de políticas e atuação de gestores públicos: experiências recentes das políticas de redução das desigualdades, IPEA.
[15] Fernandes, A. and P. Santana (2018), Fiscal Federalism in Brazil: Main issues, challenges, and reforms, OECD Network on Fiscal Relations Across Levels of Government, https://www.oecd.org/tax/federalism/reforms-of-fiscal-relations-in-brazil.pdf.
[14] Frente Nacional de Prefeitos (2020), Anuário Multi Cidades 2020 - Finanças dos Municípios do Brasil, https://multimidia.fnp.org.br/biblioteca/publicacoes/item/760-anuario-multi-cidades-2019-ano-15-2020.
[44] Furtado, C. and T. Soares (2018), Impacto da Bonificação Educacional em Pernambuco, https://doi.org/10.18222/eae.v0ix.3940.
[41] Glewwe, P. et al. (2013), School resources and educational outcomes in developing countries: A review of the literature from 1990 to 2010, Education Policy in Developing Countries, University of Chicago Press.
[57] Hirata, G. and P. Oliveira (2017), Accountability in Education: Meeting our Commitments, Brazil Country Case Study., UNESCO.
[42] Hyman, J. (2017), Does Money Matter in the Long Run? Effects of School Sprending on Educational Attainment, https://www.aeaweb.org/articles?id=10.1257/pol.20150249.
[22] IMF (2015), Making Public Investment More Efficient, International Monetary Fund, https://www.imf.org/external/np/pp/eng/2015/061115.pdf.
[5] INTOSAI (2019), GUID 3920-The Performance Auditing Process.
[37] Matias, A. et al. (2018), “Níveis de Gastos e Eficiência Pública em Educação: Um Estudo de Municípios Paulistas Utilizando Análise Envoltória de Dados”, Revista de Administração da Universidade Federal de Santa Maria, vol. 11, núm. 4, http://www.redalyc.org/articulo.oa?id=273458364008.
[56] Matijiascic, M. (2017), Professores da educação básica no Brasil: Condições de vida, inserção no mercado de trabalho e remuneração, IPEA, http://repositorio.ipea.gov.br/bitstream/11058/7929/1/td_2304.pdf.
[23] Mizell, L. and D. Allain-Dupré (2013), “Creating Conditions for Effective Public Investment: Sub-national Capacities in a Multi-level Governance Context”, OECD Regional Development Working Papers, No. 2013/4, OECD Publishing, Paris, https://dx.doi.org/10.1787/5k49j2cjv5mq-en.
[43] Moreira, A. (2017), Eficiência do gasto da educação fundamental municipal, IPEA, http://www.ipea.gov.br/portal/images/stories/PDFs/TDs/td_2308.pdf.
[6] NAO (2018), Framework to review programmes.
[7] NAO (2016), Investigation into Just Solutions International, https://www.nao.org.uk/wp-content/uploads/2016/01/Investigation-into-Just-Solutions.pdf.
[45] National Education Conference (CONAE) (2013), O PNE na articulação do sistema nacional de educação: participação popular, cooperação federativa e regime de colaboração, Brasília, DF: MEC, http://fne.mec.gov.br/images/pdf/documentoreferenciaconae2014versaofinal.pdf.
[17] OECD (2020), Feedback from TCs and the local team to the OECD.
[10] OECD (2019), Effective Public Investment Across Levels of Government – Principles for Action, OECD, https://www.oecd.org/effective-public-investment-toolkit/OECD_Public_Investment_Implementation_Brochure_2019.pdf.
[2] OECD (2019), Effective Public Investment across Levels of Government: Implementing the OECD Principles, https://www.oecd.org/effective-public-investment-toolkit/ (accessed on 16 August 2019).
[4] OECD (2019), Effective Public Investment Across Levels of Government: Implementing the Principles, OECD Publishing, https://www.oecd.org/effective-public-investment-toolkit/OECD_Public_Investment_Implementation_Brochure_2019.pdf.
[1] OECD (2019), “Making decentralisation work: A handbook for policy-makers”, in Making Decentralisation Work: A Handbook for Policy-Makers, OECD Publishing, Paris, https://dx.doi.org/10.1787/dd49116c-en.
[3] OECD (2019), Making Decentralisation Work: A Handbook for Policy-Makers, OECD Multi-level Governance Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/g2g9faa7-en.
[16] OECD (2019), OECD study mission to Brazil, December 2019.
[19] OECD (2018), Centre Stage 2: The Organisation and Functions of the Centre of Government in OECD Countries, OECD Publishing, https://www.oecd.org/gov/centre-stage-2.pdf.
[24] OECD (2018), Rethinking Regional Development Policy-making, OECD Multi-level Governance Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264293014-en.
[20] OECD (2017), Government at a Glance, OECD Publishing, https://doi.org/10.1787/22214399.
[18] OECD (2013), OECD Territorial Reviews: Brazil 2013, OECD Territorial Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264123229-en.
[26] OECD/UCLG (2019), Report of the World Observatory on Subnational Government Finance and Investment – Country Profiles, OECD, Paris, http://www.sng-wofi.org/publications/SNGWOFI_2019_report_country_profiles.pdf (accessed on 15 June 2020).
[40] Oliveira, J. et al. (2015), Educação baseada em evidências: como saber o que funciona em educação, Instituto Alfa e Beto, https://www.alfaebeto.org.br/wp-content/uploads/2015/11/Instituto-Alfa-e-.
[29] Pereira, C. and C. de Castro (2019), Educação: Contraste entre o Meio Urbano e o Meio Rural no Brasil, IPEA, http://repositorio.ipea.gov.br/bitstream/11058/9661/1/BRUA21_Ensaio5.pdf.
[30] PNUD (2016), IDHM, http://www.atlasbrasil.org.br/consulta/map.
[58] Santos, C. et al. (2017), A Dinâmica dos Gastos com Saúde e Educação Públicas no Brasil (2006-2015): impacto dos mínimos constitucionais e relação com a arrecadação tributária, IPEA, https://www.ipea.gov.br/portal/images/stories/PDFs/TDs/td_2289.pdf.
[25] Santos, R., L. Bessa and M. Lucio (2017), The Brazilian National Policy for Regional Development and the RIDE-DF Management opposite the Governance vs Brasília’s Metropolitan Area, https://doi.org/10.4000/echogeo.15072.
[50] Segatto, C. (2015), O papel dos governos estaduais nas políticas municipais de educação: uma análise dos modelos de cooperação intergovernamental, Fundação Getúlio Vargas, http://hdl.handle.net/10438/13542.
[13] Senado Noticias (2019), PEC do Pacto Federativo descentraliza recursos para estados e municípios.
[36] Tanno, C. (2017), Propostas de Aprimoramento para a Implantação do Custo Aluno Qualidade (CAQ), Câmara de Deputados, https://www2.camara.leg.br/atividadelegislativa/comissoes/comissoes-temporarias/especiais/55a-legislatura/pec-015-15-tornapermanente-o-fundeb-educacao/documentos/outros-documentos/estudo-da-consultoria-deorcamento-da-camara-dos-deputados.
[35] Tesouro Nacional (n.d.), Transferências Constitucionais e Legais, http://www.tesouro.fazenda.gov.br/transferencias-constitucionais-e-legais (accessed on 1 June 2020).
[32] TPE (2018), Todos pela Educação Já: Uma proposta suprapartidária de estratégia para a Educação Básica brasileira e prioridades para o Governo Federal em 2019-2022, https://www.todospelaeducacao.org.br/_uploads/_posts/170.pdf.
[11] Zapata, I. and C. Lafitte (2020), Multi-level governance of decentralised policies in Brazil: an analytical framework for audit assessments. Bakcgorund paper prepared for Brazilian audit institutions, (unpublished).
Note
← 1. All information used in this sub-section was retrieved from: https://www2.camara.leg.br/atividade-legislativa/comissoes/comissoes-permanentes/ce/audiencias-publicas-1/apresentacoes/apresentacao-lina-katia-ufjf-sistemas-de-avaliacao-da-educacao-basica.