Chapter 1. Strengthening the contributions of TCU to good governance in Brazil
Supreme audit institutions (SAIs) face a demanding auditing environment, where governance challenges persist and consistently present new risks. In this context, SAIs must remain flexible and responsive to changes and emerging risks in their own planning and auditing. This chapter explores opportunities for the Tribunal de Contas da União (TCU), Brazil’s SAI, to make improvements to its own strategy and operations in order to more effectively contribute to good governance in Brazil. This includes a greater focus on the institutional characteristics that position TCU to better support the policy cycle – formulation, implementation and evaluation – with evidenced-based solutions and insights that focus on long-term policy outcomes. This chapter explores steps that TCU can take to ensure it systematically upholds the principles of good governance in its own work, including its strategies, audit programming and internal mechanisms for monitoring and evaluation.
1.1. Introduction
In recent years, Brazil has experienced high tax revenues and spending, yet citizen satisfaction and trust in government remains low. There are opportunities for entities across the federal government to be more accountable to citizens by creating policies and programmes that use public resources responsibly. In addition, addressing governance challenges requires evidenced-based solutions and an openness to improve approaches across the policy cycle. The Tribunal de Contas da União (TCU), as Brazil’s SAI, plays a central role in this process, supporting policy makers and government managers to adopt policies that are effective, efficient and economical.
SAIs, regardless of their model, can contribute to more evidence-based policy making and improvements in governance. For instance, the United States Government Accountability Office (Westminster model) offers insights to reduce duplication, fragmentation and overlap in government. Portugal’s Court of Accounts (Court model) reviewed the adequacy of internal audit and control in state-owned enterprises, leading to a more systematic establishment of audit boards. The Netherlands Court of Audit (Board model) compared the quality of underlying assumptions of budgetary decisions to point to the inability of a government merger to reap the expected financial gains. TCU makes its own unique contributions to good governance, as described in subsequent sections.
The International Standards for Supreme Audit Institutions (ISSAI) highlights how SAIs can be responsive to changing environments and emerging risks, including risk assessments of the audit environment and developing a work programme that is responsive to society’s changing needs (ISSAI 12) (INTOSAI, 2013a). This chapter discusses how TCU can more closely align its work with ISSAI 12 and the selected international principles below, chosen for their pertinence to an SAI’s contributions to better governance in a complex policy making environment with cross-cutting issues. Particularly relevant principles for this chapter include:
-
Demonstrating ongoing relevance to citizens, Parliament and other stakeholders by being responsive to changing environments and emerging risks (Principle 5);
-
Enabling those charged with public sector governance to discharge their responsibilities in responding to audit findings and recommendations and taking appropriate corrective action (principle 3);
-
Being a credible source of independent and objective insight and guidance to support beneficial change in the public sector (4) and communicating effectively with stakeholders (6).
This chapter assesses TCU’s current activities in relation to these principles, and suggests opportunities for maximising TCU’s contributions to more efficient, effective and economic policy making in Brazil. In general, the chapter illustrates how TCU has a strategic planning process that is forward-looking and comprehensive, but opportunities remain for further operationalising concepts of good governance and ensuring that TCU operations are well aligned with the highest governance risks.
1.2. Overview: Good governance in Brazil and the role of TCU
Good governance through better policies, programmes and services in Brazil
To understand how TCU can better contribute to good governance, it is first necessary to define the term for purposes of this report. “Good governance” is the optimisation of the institutional arrangements and processes used to formulate, implement and deliver policies, programmes and services to benefit citizens and society. It is the determinant of policy outcomes that bear impact on economic growth, productivity, equality and environmental management, amongst other areas needed for overall well-bring. Good governance assures that the mechanisms driving the policy cycle in formulation, implementation, and monitoring and evaluation are operating effectively, efficiently and economically, and are underpinned by broader governance principles such as rule of law, transparency, integrity and accountability (OECD, 2016a).
Improving policy-making processes towards good governance, and thus outcomes, relies on a whole-of-government approach from actors across branches of power, as well as civil society. This refers to the institutional structures and networks that enable the state to achieve coherence in strategy, policy and purpose, without inefficiencies, fragmentation, redundancies and overlaps. It requires adequate systems of co-ordination and checks and balances. This includes a strategically agile Centre of Government (CoG), a functioning Legislature and independent external and internal control. Collectively, over time, entities that make up the CoG should articulate and implement a long term, big-picture strategic vision for the country aimed at sustaining prosperity and well-being for its citizens, the economy and society.
A spotlight on the Centre of Government and co-ordinated approaches to good governance
To ensure that public expenditure is used to drive improvements in economic and social outcomes, the CoG has a unique position and responsibility for co-ordinating government to this end. The CoG is the administrative structure that serves the Executive (President or Prime Minister, and the Cabinet collectively). Often the CoG is made up of more than one unit, fulfilling different functions. The CoG can include other core institutions at the “Centre” that directly support whole-of-government functioning, and usually exercise some oversight of ministries’ activities.1 CoGs in OECD member countries fell into the spotlight during fiscal and economic crises of the last decade. In a context of multidimensional policy issues, CoGs are increasingly responsible to ensure a more co-ordinated approach to governance (OECD, 2014a).
In Brazil, the institutions that generally constitute the CoG consist of the Presidency (Casa Civil), the Ministry of Finance (Ministério da Fazenda, MF) and the Ministry of Planning, Development and Management (Ministério do Planejamento, Desenvolvimento e Gestão, MP) (TCU, 2016a). Within the MF and MP, there are specialised secretariats and units that are responsible for steering and policy setting related to different governance functions. For instance, within the MP, the Federal Budget Secretariat formulates the annual budget proposal and in the MF, the National Treasury Secretariat takes the lead on the accounting and control of the budget execution. The role of the CoG in particular policy functions is explored in this report to understand how TCU can use audit and advisory work to strengthen the centre’s effectiveness and efficiency (OECD, 2014a).
In 2014, Brazil’s government effectiveness was rated the low score of 47/100, a trend that is worsening with political and economic crises (World Bank, 2014). Given the substantial sum of money dedicated to high priority areas like education and health, mentioned below, improving effectiveness and efficiency of government expenditure can have a significant impact on the overall development of the country and the quality of life for the average Brazilian. Particularly in the face of fiscal austerity and persistent economic recession, policy makers need to know what works and what does not in order to ensure quality delivery of services. For this, meaningful monitoring and evaluation should feed into policy formulation so that entities can adjust for improvements.
Interconnected policy challenges, and “wicked problems,”2 require governments to ensure communication and co-ordination as well as budget sustainability to implement solutions over the medium and long-term. For example, Brazilian citizens indicated, in 2016, that protecting the environment is a priority social issue to be resolved (Edelman, 2016). Improvements have been made in environmental protection, but successes are impeded by a lack of coherence between key sectors for environmental protection, including water and agricultural sectors (OECD, 2015a; Sabogal et al, 2016). Cross-governmental, sustainable and inclusive action is needed for other complex issues, such as public safety, population ageing, climate change and continued eradication of poverty in Brazil. Moreover, the complexity of the Sustainable Development Goals (SDGs) will challenge governments to better understand the spill over effects of certain policies on others.
In Brazil, tax revenue is high but citizen satisfaction and trust in government is low
In recent years, policy making in Brazil has been characterised largely by high tax revenue and increases in public expenditure, particularly in social areas. Tax revenues constitute 33.4% of Brazil’s GDP in 2014, the highest of all Latin American and Caribbean countries (OECD, 2016d). Brazil’s public administration has had a high level of public expenditure in recent years, as exemplified by select sectors. For instance, Brazil’s expenditure on educational institutions rose as a percentage of GDP by 4% between 2010 and 2012, while the average in OECD countries was a decrease of 3% (OECD 2016d). By 2016, Brazil’s investments in educational institutions amounted to 5.6% of the GDP, with an OECD average of 5.3% (OECD 2016d).3 Health spending and social security contributions have also claimed a large share of public expenditure, constituting 8.2% (World Bank, 2015) and 8.7% of GDP, in 2014 respectfully, while the latter average in the Latin American and Caribbean region was 3.7% (OECD, 2016d).
Nonetheless, increases in spending have not always led to perceived improvements in service delivery. Despite the increases in expenditure in particular areas citizen satisfaction has decreased with both education and health more than ten percentage points from 2007-2014, to 45% and 32% respectively (OECD, 2015b). Moreover, levels of education and skills are lower, on average, in Brazil than in OECD member countries, including Mexico and Chile, despite Brazil’s higher investment in education (Figure 1.1). Brazil has higher than average ‘life satisfaction’4 and is at par with OECD member countries for civic engagement and governance (Figure 1.1).
Source: OECD (2016d), «Better Life Index 2015», OECD Social and Welfare Statistics (database), OECD Publishing, Paris https://doi.org/10.1787/data-00823-en as cited in OECD (2015c), OECD Economic Surveys: Brazil 2015, OECD Publishing, Paris https://doi.org/10.1787/eco_surveys-bra-2015-en.
A state that is responsive to citizens’ needs ensures that its functions and supporting activities are working well. Effective governance arrangements can restore citizens’ trust in their government’s ability to develop and implement policies that lead to sustainable and inclusive growth. It can also position governments to address weaknesses, at the roots, that are uncovered during crisis moments. This means demonstrating value for money – or effectiveness, efficiency and economy – with limited public resources to deliver the best possible outcomes (OECD, 2016a).
In addition, OECD experiences show that restoring trust in government is important for strengthening integrity in society, such as paying taxes, promoting adherence to regulations and fostering greater public security (OECD, 2015b). From 2014-2015, Brazil’s rating in the Corruption Perception Index decreased from 43 to 38, which constituted the biggest deterioration in public trust on the index that year (TI, 2015). This decline in trust coincides with a fall in the legitimacy of the state from 5.1 in 2015 to 5.9 on the 2016 Fragile State Index (FFP, 2016). Trust in government also can affect investment opportunities that come with a more stable, transparent and productive public administration. Optimising government arrangements and processes is important in the face of mistrust in government.
Through their audits, evaluations and advisory work, of cross-cutting issues in particular, SAIs can play a critical role in restoring citizens’ trust in government. Constitutionally independent from the executive branch, SAIs bring a unique horizontal view of the functioning of government. As demonstrated in OECD’s report, “Supreme Audit Institutions and good governance: Oversight, Insight and Foresight” (2016) SAIs are using this vantage point to assess policy-making processes at every stage of the policy cycle: formulation, implementation and evaluation. To do so effectively requires high-level commitment, as well as strategic and operational changes within an SAI. This chapter focuses on recommendations (see Table 1.1) for TCU to ensure its relevance and responsiveness to a changing environment, and to remain a credible source of information for citizens, auditees and decision-makers.
1.3. Ensuring a systemic, long-term and citizen-oriented view
TCU could ensure that the formulation of its strategic plans enable it to tackle systemic and emerging risks to society over the medium and long-term.
SAIs that seek to improve their contributions to good governance – or more efficient, effective and economical public policies and programmes – should also ensure their own institutions are governed well. This begins at the strategic level. International standards relevant to SAIs can offer a framework for understanding how SAIs are strategically aligning themselves with principles of good governance. For instance, the United Nations Resolution 69/228, emphasises the need for strengthening SAIs to promote and foster the efficiency, accountability, effectiveness and transparency in government (United Nations, 2014). In addition, ISSAI 12 notes, “objectives and principles [that] are intended for SAIs to strive towards and to enable all SAIs to communicate and promote the value and benefits that they can bring to democracy and accountability in their respective jurisdictions” (INTOSAI, 2013a).
Table 1.2 elaborates on select principles of ISSAI 12, and shows linkages to the mission and priorities of TCU’s current strategic plan for 2015 to 2021. TCU’s strategic planning captures much of the spirit of ISSAI 12. This underscores the importance TCU places on activities that go beyond its judicial function, and include other mandates in service of the Congress and Brazilian citizens. For example, TCU conducts evaluations and offers recommendations for improvement in public administration (TCU, 2016b), and identifies vulnerabilities in the internal control systems related to public procurement, contracting and public-private partnerships (Santiso, 2009). Moreover, TCU’s Systemic Reports (i.e. Fiscs), based on surveys that TCU carries out in a range of sectors, such as health, education, social welfare and information technology, offer an aggregated view of TCU’s work in these areas, emphasising challenges, risks and potential solutions. Annex 1.A1 further demonstrates how TCU has incorporated good governance elements into its various strategic documents.
As reflected in its strategic plan for 2011 to 2015, TCU has focused efforts in strengthening the ‘effectiveness’ component of its mission by promoting improvements in governance through use of performance audits, advice and consultations. The strategic plan includes concrete activities that are critical for SAIs to effectively contribute to good governance, including risk-based analyses, systemic diagnoses of government programmes, timely resolution of cases of wrongdoing and various actions to enhance management and innovation. For instance, TCU’s strategic objective 5 focuses on conducting surveys in relevant areas in order to promote the assessment of the governance and management of public policies and contribute to the reduction of bureaucracy (TCU, 2015b). TCU has allocated 17% of external control resources to this objective, which is amongst the highest allocation for external control activities (TCU, 2015a).
In addition, TCU’s strategic planning process is comprehensive and includes scenario planning in order to adopt a broader view of horizontal, medium and longer-term governance challenges. It seeks to reflect the insights of all stakeholders, including civil society in addition to the executive branch and the Congress. Further information about TCU’s strategic planning process is provided in Box 1.1. The approaches adopted for the 2011-2015 Strategic Plan were carried forward for the planning process of the 2015-2021 Strategic Plan (Plano Estratégico do Tribunal, PET). The predominant steps of the current PET were (i) an institutional diagnosis of internal and external perceptions of TCU’s work, as well as an assessment of the political, economic, social, technological, legal and administrative environment, (ii) identification of strategic drivers for TCU’s work, and (iii) elaboration of a strategic mapping that appears in the PET (TCU, 2016a).
TCU’s strategic planning involves: i) clarifying the organisation’s mission and values; ii) developing a vision for the future; iii) analysing external opportunities and risks; iv) assessing internal strengths and weaknesses; v) evaluating alternative strategies; and vi) developing monitorable action plans. Since the first TCU Strategic Plan (Plano Estratégico do Tribunal, PET) in 1994, seven strategic plans have been issued by TCU, each varying in their approach.
TCU’s 2011-15 Strategic Plan was developed using a balanced scoreboard approach and scenario analysis, driven by an awareness of the growing complexity of audit environment. Seeking to align with other supreme audit institutions, TCU wanted a strategy that would be effective in the long term and that would not become obsolete quickly.
To formulate a strategy that incorporated potential changes in the environment that could impact TCU’s work, TCU used a scenario analysis based on the “Grumbach method”. The Grumbach method is used to support the formulation of strategies by organisations that find themselves in a competitive and turbulent environment, where traditional models of planning, forecasting and projections are not appropriate. This approach applies traditional tools of strategic planning together with scenarios planning methods, for example, Delphi, Cross-Impact and the Monte Carlo Simulation.
The Grumbach method closes the strategic management cycle by prioritising and planning the strategic initiatives and their monitoring. Among the issued examined in the scenario planning were:
-
macro and micro-economic trends within the Brazilian economy
-
take up of e-government and m-government
-
participation of society in the oversight of public expenditure
-
private participation in the delivery of public services
-
increasing outsourcing in government service delivery
-
ability of the public administration to attract and retain officials
-
actions to prevent and detect corruption within the public sector.
TCU used a consultative process, informing the scenario analysis with a survey of 1,500 external experts on the economy, the environment and through direct citizen engagement, as well as surveys and consultations across TCU.
Source: Adapted from OECD (2014b), Chile’s Supreme Audit Institution: Enhancing Strategic Agility and Public Trust, OECD Public Governance Reviews, OECD Publishing, Paris. https://doi.org/10.1787/9789264207561-en
The 2011-2015 PET (as described in Box 1.1) gave thoughtful consideration to longer-term trends that would impact TCU’s work, yet TCU could take additional steps to ensure that scenario planning is based on the highest risks for society. Principle 5 of ISSAI 12 suggests that to provide value and benefits for citizens, SAIs should remain responsive to a changing environment and emerging risks (INTOSAI, 2013a). To further align with this principle, TCU could ensure that medium and long-term risks for society also steer the External Control and Guidelines (2-year basis), as well as the annual plans of the Secretary General of External Control (Secretária-Geral de Controle Externo, SEGECEX), described further below. More specifically:
-
TCU could further and more systematically integrate medium and long-term policy issues into its new risk-based approach to audit programming to ensure that its activities are responsive to society’s needs and aligned with TCU’s strategic plans.
-
TCU could ensure that audits integrate governance elements, and serve as a vehicle to collect information on broader governance issues and national goals that are systematic and cross-cutting.
TCU could further and more systematically integrate medium and long-term policy issues into its new risk-based approach to audit programming to ensure that its activities are responsive to society’s needs and aligned with TCU’s strategic plans.
Ongoing social, political and economic changes in Brazil create a challenging auditing environment, which places high demands on TCU to ensure its continued relevance and impact. Process to understand this environment should ultimately help to TCU to concentrate limited resources on critical, high-risk areas. TCU’s existing strategic documents provide a foundation for taking a longer-term, risk-based perspective in fulfilling its mission and objectives. As mentioned above, TCU’s strategic planning process includes scenario planning to ensure that the Strategic Plan, usually covering a 5 or 6 year period, enables TCU to remain relevant to society’s issues. However, TCU staff indicated that the implementation of the Strategic Plan is difficult in practice, and that annual plans are not always reflective of aims set out in either the Strategic Plan (6 year) or the External Control Plan (2 year).
According to TCU officials, one factor that complicates this issue is the increasing number of Congressional requests5 for supervisory activities made in-year in addition to, or in place of, those laid out in the 2-year External Control Plan or in the annual plans of SEGECEX secretariats. Such requests may come from Minister rapporteurs or Congress, the latter of which have been on the rise as a percentage of total activities in recent years (TCU, 2016c, d; 2015a; 2014a). Some SAIs face similar challenges. For instance, at the U.S. Government Accountability Office (GAO), required and requested audits comprise approximately 90% of its work (OECD 2016a). Ensuring impact and relevance for tackling key policy challenges meant effectively prioritising within its audit portfolio, in part, through discussions and consultation with Congressional requesters, while still maintaining autonomy in its audit programming and selection of auditees. Engaging requesters in advance, like the GAO, could help TCU to balance immediate priorities with medium and long-term goals.
Interviews with TCU staff suggest room for improvement in addressing challenges and emerging risks on an annual basis. The External Control Plan and Guidelines (“Diretrizes”) at the operational level inform the preparation of annual plans of each unit at the tactical level (TCU, 2016a). The annual plans are prepared at the secretariat level and include proposals for audits and initiatives, aligned with the Strategic Plan. According to TCU staff, approximately 90% of audits are proposed by the secretariats with the technical knowledge on the audit subject and the other 10% comes from requests, namely from Congress for supervisory proceedings. Secretariats make these proposals to TCU’s Ministers, who have an overarching view of all TCU’s activities and can consult rapporteurs formerly assigned to the sector or audit subject.
In interviews and workshops, TCU officials expressed concern that there is too much flexibility for annual plans of individual units to direct resources to areas without consulting internally or comparing to the work of other units. This can lead to overextension or lack of coherence between TCU units, which can ultimately undermine broader strategic objectives. In 2015, more resources were channelled towards certain objectives (e.g. objectives 4, 8 and 10, as described in Annex 1.A1) than had been originally anticipated, and would require readjustment in coming years if annual plans are to meet the objectives by 2021.
TCU is exploring the integration of a more rigorous risk-based approach to audit programming on an annual basis. TCU resolution (Res 269/2015, art 19) requires that audit programming involves the selection of audit topics to be based on risk criteria, materiality, relevance and opportunity, but this is not yet fully in practice (TCU, 2015c). TCU has launched an initiative to develop a methodology to comply with this regulation called Method for Selection of Objects and Control Actions (Método de Seleção de Objetos e Ações de Controle). In 2015, TCU began this by assessing whether it is addressing the most relevant issues through its audits.
TCU can build on these efforts to adopt risk-based approaches to make planning of programmes and resource allocation more systemically focused upon emerging risks and medium to long-term policy issues. Basing audit programmes on a risk-analysis will help to ensure (i) that TCU’s activities are based on cross-cutting policy issues and priorities for society; and (ii) resources are allocated in accordance with identified institutional priorities articulated in various plans. TCU has a broad scope of engagement across a wide variety of subject matter and governance functions, but it cannot provide complete assurance on government performance or compliance. A strategic, risk-based approach helps TCU to optimise the allocation of finite audit resources. Adopting more rigorous risk-based audit programming that focuses on risks to government and to society at an aggregate level could enable TCU to more closely align resource allocation to the most pressing needs.
In addition, a systematic risk-based approach to audit programming can help ensure that short-term initiatives do not detract from TCU’s efforts to induce change in governance functions over the medium and long-term. Integrating a risk-based approach into individual audit programmes of units would serve to align TCU’s efforts across Secretariats, and would provide SEGECEX with a more robust and consistent framework to assess proposed annual plans of SEGECEX units. Moreover, the annual plans themselves would be geared more towards the most pressing, cross-cutting needs and highest risks in government. A risk-based audit programme would act as a guide when developing such plans and during internal deliberations on the scope and focus of the portfolio and individual audits.
TCU could also apply this more rigorous risk-based approach to the 2-year External Control Plans. Such systematic risk-based audit programming can contribute to the evidence-based use of resources and prioritisation of policy objectives that are forward looking. Risk-based programming provides a framework of analysis that TCU officials can refer to for internal agreement of audit priorities. Moreover, in addition to facilitating the selection of audits and allocation of resources, a risk-based approach to audit programming can aid SAIs in communicating its decision making internally and externally in order to ensure that salient issues are brought to the attention of managers and relevant stakeholders. This also helps to strengthen the transparency of the SAI and support its justification for any actions its takes. For practical implementation of a risk-based approach that integrates emerging and medium to long-term risks to society, TCU could first look to the general steps for developing a risk-based audit programme, as illustrated in Figure 1.2.
Source: National Audit Systems Network (2014) ‘Planning for Audits of Official Control Systems’ Version 1, February 2014, http://www.livsmedelsverket.se/globalassets/produktion-handel-kontroll/vagledningar-ontrollhandbocker/vagledningar-och-information-fran-eukommissionen/risk-based-planning-for-audits-of-official-control-systems-february-2014
Figure 1.2 highlights the importance of “data and information” as inputs (stage 2.1) to risk-based audit programming, which can involve considerations of the national government agenda and priorities. Considering such information as inputs into its process can help to ensure that risks are reflective of emerging challenges and long-term policy goals. This would help TCU to be proactive in it is audit programming, going beyond auditing and oversight that is reactive to short-term challenges. To inform its risk-based programming, TCU can draw from its audits and evaluations for information about broader governance issues, as well as the obstacles for improving policies and programmes. Box 1.2 illustrates further the sources and activities for improving inputs into its risk-based audit programming for strengthening the insight and foresight offered by TCU’s audits.
TCU would benefit from having a deeper understanding of medium and long-term challenges to improving the governance and management of public administration, such that its audit work can promote lasting improvements in public administration. This approach is important to ensure that TCU can continue looking ahead, while government is more focused on short-term responses to crisis matters. To anticipate medium and long-term risks, which can then be integrated into its audit programming, TCU could generate insight and foresight activities to better understand the current and future audit environment. Table 1.3 provides examples of SAIs activities in this respect. Such foresight is particularly important where Executive Branch entities may be more limited in perspective by political cycles.
For deepening TCU’s understanding of medium and longer-term challenges to governance and thus to society it can integrate and build from the work that TCU has done previously. For instance, TCU surveyed public officials in 2014 on their capacity for leadership, for strategy and for control. These elements are in line with international indicators on Executive Branch Effectiveness from the World Governance Indicators Project (WGI, 2017). Such information provides TCU with a baseline understanding of the maturity of public official’s capacity, and where risks may appear.
TCU can also look to internationally comparative data, such as OECD’s Better Life Index, the Social Progress Imperative and the World Happiness Report, where indicators provide a commentary on a range of social and economic outcomes. More consistent reference to TCU’s own General Governance Index (IGG) can also be useful in diagnosing challenges in the audit environment (TCU, 2016e). Other sources could include the World Governance Indicators, the Social Progress Index and eventual progress measurement of Brazil against the SDGs. Such indices can offer a snapshot for TCU to better understand its future environment. To strategically align itself with ISSAI 12, such data sources and indicators can be incorporated into its risk-based audit programming to complement their use during individual audit engagements.
Scanning such material annually, and anticipating risks of the future, would provide TCU with a more systematic analysis of short, medium and long term challenges that the country currently, or could, face in the future. There is an opportunity to make this understanding consistent and systematic, by integrating elements into TCU’s new risk-based approach to audit programming. TCU officials noted that TCU has been working for two years on developing these approaches, with principles of risk being widely adopted for the 2017 planning cycle.
Source: OECD (2016a), Supreme Audit Institutions and Good Governance: Oversight, Insight and Foresight, OECD Public Governance Reviews, OECD Publishing, Paris. https://doi.org/10.1787/9789264263871-en; Social Progress Imperative, http://www.socialprogressimperative.org/; World Bank World Governance Indicators Project, http://info.worldbank.org/governance/wgi/index.aspx#home; World Happiness Report, http://worldhappiness.report/
Criteria for risks can be drawn from different sources and can change depending on the type of audit. For example, material errors in the accounts of the programme can be a main criterion for prioritising risks related to financial audits. For performance audits, criteria can focus more on elements of effectiveness, efficiency and economy, and the extent to which the policy or programme is achieving its goals. Time elapsed since the last audit, or the occurrence of major changes in the audited entity, are other examples of criterion. Use of criteria for assessing risks facilitates ranking of audit priorities, according to the perceived impact and probability of the risk posed by a particular audit subject. Box 1.3 provides examples from countries with risk-based audit programming.
The approach of supreme audit institutions (SAIs) to audit programming depends, in part, on their mandate and their level of discretionary authority and independence. Nonetheless, audit institutions can take different approaches to audit programming. One example is the “cycle-based” approach, whereby all entities in an audit universe (i.e. the range of organisations and activities that can be audited) are audited in year-based cycles, such as every 3 to 5 years. This approach ensures a total coverage of the audit universe within a certain timeframe, yet some audits may have limited or no value. Another approach is the “audit-on-demand” or “incident-based” audit programming. This approach involves responding to the demands or requests related to specific matters or incidents. For instance, in some countries, SAIs respond to specific requests of the Parliament, and they reserve resources for this purpose.
The “risk-based” approach focuses audit capacity and efforts on key risk areas of the audit universe. For SAIs facing an expanded audit mandate, like Mexico’s SAI (the Auditoría Superior de la Federación, or ASF), a risk-based approach facilitates prioritisation of audits and targets resource allocation based on a qualitative or quantitative (or both) assessment of the audits’ added value. This approach can be borne from necessity in a complex, resource-constrained environment, where it may not be realistic for SAIs to audit all entities, programmes or policies. Moreover, international standards for auditing call for SAIs to take a risk-based approach when selecting audits. For instance, International Standard for Supreme Audit Institutions (ISSAI) 300 on Fundamental Principles of Performance Auditing notes:
Auditors should select audit topics through the SAI’s strategic planning process by analysing potential topics and conducting research to identify risks and problems… The topic selection process should aim to maximise the expected impact of the audit while taking account audit capacities (e.g. human resources and professional skills). Formal techniques to prepare the strategic planning process, such as risk analysis or problem assessments, can help structure the process but need to be complemented by professional judgement to avoid one-sided assessments.
Risk-based audit programming and planning in SAIs can vary, but it generally involves a process of criteria development, risk identification, analysis, scoring and mapping, prioritisation and selection of audits in relation to strategic objectives, capacity and resources. Criteria selection can be drawn from different sources, as illustrated in the examples below. For instance, criteria can change depending on different type of audits. For financial audits, material errors in the accounts of the programme can be a main criterion. For performance audits, criterion can focus more on elements of effectiveness, efficiency and economy, and the extent to which the policy or programme is achieving its goals. The risk map and result of the assessment provides an overview of risks, and the ranking reflects criteria linked to the perceived impact and probability of the risk, such as the length of time since the last audit or the occurrence of major changes in the audited entity. The following are examples from countries with risk-based audit programming.
Belgium
The Belgian Court of Accounts uses a ‘Financial Operational Risk Model’ (FORM), which is an integrated risk-based planning model. The prioritization of audits is based on three criteria, one of which is the financial-budgetary significance for evaluating the budgetary relevance of the institution on a consolidated level. Budget thresholds are determined taken into account the total federal budget. The second criterion relates to budget priorities and controls. As part of this, the Court considers policies as well as horizontal issues affecting different public entities. The third criterion involves general considerations based on the professional judgement of the Court’s management. This criterion receives less weight for programmes with smaller budgets.
Denmark
The Danish Rigsrevisionen uses a risk-based approach at the audit-planning level. The purpose of the analysis is to obtain insights into the department’s situation and to uncover potential risk areas. Initially, Rigsrevisionen carries out a strategic analysis of each ministerial area and department. The strategic analyses of the departments also include reviews of the underlying agencies and public bodies and their management. Subsequently, a similar strategic analysis is carried out on in the public body. Before the audit is launched, Rigsrevisionen discusses the strategic analysis with the management of the department and the relevant public body to reduce the risk of overlooking significant areas in the audit. The strategic analysis provides the basis for planning the audit, which is focused on the identified risk areas to ensure that required audits are carried out and audit opinions on the accounts can be issued.
The European Court of Auditors (ECA)
The ECA has a yearly policy and risk review to take into account issues like developments in policy areas, stakeholders’ priorities, results of recent audits, the work of other audit institutions and media coverage. This review is done in the form of ‘policy scans’ for each major policy area and helps ECA to define priorities and align them with strategic objectives. The ‘policy scans’ inform various audit chambers to prepare lists of proposed audit tasks and information on their relative priority. After consideration and approval of the tasks, ECA’s annual work programme sets out the audit tasks to be implemented during the year and the resources allocated to them. The programme is reviewed and updated during the year to take into account any changes to priorities. Performance and compliance audit topics are selected at the ECA’s discretion on the basis of criteria like such as risk of irregularities, poor performance, potential for improvement and public interest.
Source: Lonsdale, J. et al (2011), Performance Auditing: Contributing to Accountability in Democratic Government, Edward Elgar Publishing, Cheltenham, United Kingdom; Rigsrevisionen (Danish SAI) (2016), “The strategic analysis”, 17 August, http://uk.rigsrevisionen.dk/how-we-audit/annual-audit/strategic-analysis/; European Court of Auditors (2015), “Work programme”, www.eca.europa.eu/Lists/ECADocuments/WP2015/WP2015_EN.pdf; INTOSAI (2004), “ISSAI 300: Fundamental Principles of Performance Auditing”, http://www.issai.org/data/files/33/45/70/E6/F93E7510D33F2E75CA5818A8/issai-300-english.pdf.
TCU could ensure that audits integrate governance elements, and serve as a vehicle to collect information on broader governance issues and national goals that are systematic and cross-cutting.
In the last few years, TCU has invested resources into developing a more holistic view of governance, both horizontally across ministries and vertically between levels of government together with state audit institutions (Tribunals de Contas dos Estados, TCEs). For instance, SEGECEX has developed products that take a more holistic look at the functioning of a particular sector (e.g. Fiscobras and FiscSaude), and Secretary of Government Macro-Evaluation (Secretaria de Macroavaliação Governamental, SEMAG) co-ordinates inputs of different units for the drafting of the year-end opinion on the President’s consolidated accounts. In addition, TCU has conducted cross-cutting audits that assess the maturity of particular governance functions, such as the maturity of risk management across government (see Chapter 4), or the IT audit on integration of information technology across government. The latter was praised by Executive representatives consulted for this study for its usefulness in highlighting areas for improvement.
This work illustrates concrete steps TCU has taken to integrate governance elements into its audit and evaluation work; however, TCU has yet to consistently apply criteria that would facilitate comparability on systematic issues across audits over time. When audits are more intentionally geared towards identifying challenges and solutions for cross-cutting policy issues, it allows for easier tracking of improvements and it facilitate coherence among policies and programmes. Moreover, a systematic approach can provide better inputs to inform TCU’s risk-based audit programming. One step TCU can take is to synthesise issues raised across its work streams, products and services, such as audit reports, contract assessments, asset declaration analysis and fielding of citizen complaints. As part of this effort to address systemic, cross-cutting issues, TCU could further define what should be considered of “systemic importance” to ensure coherence between assessments on this issue, and communicate effectively across the organisation. TCU could make such audits cyclical, reviewing on a systematic basis that is pre-determined and planned for in TCU’s annual audit plans.
In addition, TCU could strengthen its approach to supporting good governance by more systematically looking upstream to policy formulation, implementation and evaluation of policies and programmes being audited – not simply to the results of policies and programmes. For instance, audits could further integrate criteria that hold government to account for improved governance and policy making (discussed further in Chapter 2 through 5 for specific governance functions). Moreover, opportunities remain for TCU to leverage findings from its variety of accounting and supervisory proceedings in order to highlight areas of importance to the CoG in the execution of their duties.
TCU’s own Framework to Assess Governance in Public Policies (2014) and Framework for Evaluation of the Centre of Government (2016) offers a starting point for further institutionalising its audits and evaluations of systemic issues. As a supplement to these frameworks, audit teams can look to the international principles promoted in subsequent chapters of this review. Further, OECD’s (2016) comparative review entitled “Supreme Audit Institutions and Good Governance: Oversight, Insight and Foresight”, maps international governance principles around key stages of policy making, which can serve as a reference tool for audit teams in developing audits that will better inform the maturity of governance (OECD, 2016a).
1.4. Improving TCU’s capacity and co-ordination to remain strategically agile in a changing audit environment
TCU could continue strengthening its capacity and internal co-ordination to improve its horizontal view on cross-cutting issues and remain responsive to emerging risks.
ISSAI 12 calls for SAIs to lead by example in their work, which includes ensuring good governance of the SAI itself by adopting and complying with good governance principles (ISSAI 12, principle 9.1) (INTOSAI, 2013a). To achieve this, SAIs faced constant demand for greater efficiency, effectiveness and economy in a changing audit environment. Not only do the policy issues of the audit environment change, but so do the processes through which policies and processes are developed. In turn, this can require upskilling of those who are required to audit such policies. Indeed, INTOSAI’s General Standards in Government Auditing (ISSAI, 200) require SAIs to: develop and train SAI employees to enable them to perform their tasks effectively, and to define the basis for the advancement of auditors and other staff (INTOSAI, 2013b).
TCU has taken steps to adapt its strategy and modernise its operations in order to better support good governance and drive improvements in the performance of public administration. These actions include, amongst others, (i) internal reorganisation and specialisation of units, (ii) a decrease in the number of supervisory proceedings, with a greater emphasis on going into greater depth in the audit subject; (iii) improving the balance between compliance and performance audits, including shifting the allocation of resources to the latter (see Figure 1.3); and (iv) an evolved vision and elaborate strategic planning process, as discussed above (TCU, 2016d; 2015a). As noted in Figure 1.3, despite the relatively equal provision of HD to both compliance and performance audits, the output of compliance audits remained higher in 2015. The 104 performance audits undertaken in 2015 required 45.5% of HD, whereas the 242 compliance audits required 52.5% (TCU, 2015a).
Overall, fewer audits are being undertaken at TCU than previously. Yet emphasis is shifting towards performance audits that are more labour-intensive. As TCU continues this shift, and as it considers strengthening its contributions to good governance, it could consider how it will “support the skills and experience available within the SAI and identify the skills which are absent; provide a good distribution of skills to auditing tasks and assign a sufficient number of persons for the audit,” as promoted by ISSAI 200 (INTOSAI, 2013b).
Source: TCU (2015d), Report of the Secretary General of External Control (Relatório da Secretaria-Geral de Controle Externo), Brasilia, accessed July 2017. http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A2572667DE0157298FA4C94D16&inline=1
To provide value and benefits to society, TCU could continue to build the capacity and co-ordination needed to take a more horizontal view on cross-cutting issues and to remain responsive to emerging risks to society. To this end, TCU could consider the following actions:
-
TCU could improve the coherence and effectiveness of its work on cross-cutting, high-risk governance issues by strengthening integration of audit teams and internal co-ordination mechanisms; and
-
TCU could build capacity and knowledge for conducting audits to identify emerging risks and challenges facing government and society.
TCU could improve the coherence and effectiveness of its work on cross-cutting, high-risk governance issues by strengthening integration of audit teams and internal co-ordination mechanisms.
To better position itself for addressing cross-cutting issues that affect multiple government programs and policies, TCU could take steps to further integrate its work internally that touch on such issues. This could include: (i) reducing the level of fragmentation or siloes between audit units and (ii) optimising information-sharing and resources across audit activities. TCU officials highlighted the need for reducing siloes or fragmentation between department in order to avoid duplication and to ensure coherence of findings on similar topics.
TCU reorganised internally by relevant sectoral areas as part of the 2011-2015 PET. It created four thematic co-ordination offices in SEGECEX (social issues, infrastructure, development and essential public services). The reorganisation was aimed at greater specialisation of audit staff, enabling each new secretariat to better identify its own risks and relevant situations, as well as to better understand the models and tools of governance that are required for success in those domains (OECD, 2014b). According to TCU officials, the reorganisation is still a work in progress, and they highlighted siloes that still exist between staff.
To reduce siloes and strengthen coherence of work on cross-cutting issues, TCU could consider structural changes that focus on select whole-of-government challenges. These departments or teams can help to provide a horizontal view of cross-cutting governance and policy issues. For instance, TCU recently established, in early 2017, a new department of Institutional Relations on Fight Against Fraud and Corruption (Secretaria de Relações Institucionais no Combate à Fraude e Corrupção - SECCOR). The purpose of SECCOR is to develop, encourage, monitor, support and co-ordinate control actions to combat fraud and corruption, by strengthening the relationship between TCU and the other control and inspection bodies and entities, as well as capacity building for obtaining, analysing and processing information (Article 14) (TCU, 2017).
SECCOR provides a level of centralisation around a specific theme that arises across all SEGECEX secretariats. TCU could dedicate additional resources, or simply assign working groups with appropriate authority, to provide a centralised repository for other systemic issues (e.g. monitoring and evaluation across government with regards to the Sustainable Development Goals). This would also help to preserve institutional memory and build expertise related to long-term, systemic problems.
In addition, finding greater synergies across TCU requires effective mechanisms to stimulate information sharing and ensure consistency of findings. For example, TCU could consider integrating units’ annual work plans to facilitate information sharing between actors who have previously worked on different audits or themes, and have transferable expertise relevant to audit subject. Moreover, pooling the wealth of expertise in a more systematic way would help to build a broader view of policy challenges and solutions. For example, greater information sharing during the design of audit engagements between different specialists in TCU, such as those working on audits and investigators of public contracts, may enable TCU to flag worrisome trends or issues that can then help to shape the scope of a review. Similarly, for example, strategic interaction between those working on governance of health policies with governance of education policies could increase the likelihood that cross-cutting or systemic issues or vulnerabilities in government are more readily detected.
One way to formalise such interactions is to employ «matrixed» audit engagements in a more strategic, systematic way. Matrixed engagements are where staff from different units work on the same audit. These audits often address cross-cutting or diverse issues that require a range of subject matter and technical expertise. Officials said TCU uses this approach, but in an ad-hoc manner. TCU could apply this methodology more systematically, as matrixed engagements allow joint development of criteria and indicators. This is also meant to ensure that work is not duplicated and that messages of TCU are consistent and coherent. More consistent use of matrixed engagements would help to overcome the challenges that teams often have in building a broad view of the audit objective, particularly on complex and cross-cutting audit subjects.
TCU could build capacity and knowledge for improving the extent to which audits identify emerging risks and challenges facing government and society.
The General Standards in Government Auditing (ISSAI 200) require that SAIs should develop and train their employees to enable them to perform their tasks effectively, and to define the basis for the advancement of auditors and other staff (INTOSAI, 2013b). How an SAI defines whether an auditor can do its work effectively can change. As discussed above, having a keen awareness of the emerging risks to governance and society should also necessitate a changing evaluation of the skill sets required to tackle such risks. ISSAI 200 further suggests that SAIs prepare manuals and other written guidance and instructions concerning the conduct of audits (INTOSAI, 2013b).
In addition to auditing manuals, such as TCU’s Performance Audit Manual (2010), TCU has recently developed frameworks for audit teams on cross-cutting subjects including assessment of public policies, IT, and Centre of Government, for audit teams to consider in conducting an audit. The frameworks have been welcomed for their promotion and dissemination of good practices and international principles, but there is uneven use of the contents across TCU. TCU could build on these efforts by strengthening their dissemination through trainings, and emphasising how to integrate key elements into individual audits. To the extent applicable, TCU could incorporate further guidance in various manuals (e.g. performance audit manual) and internal documents that are meant to aid auditors during the audit planning phase. The guidance could help auditors in a practical manner when evaluating issues that are potentially systemic, such as questions related to specific governance topics to consider when interviewing executive branch entities.
TCU provides its staff with ongoing training and development opportunities in line with international standards. Much of the course work is online. TCU staff pointed to the benefits of also having in person and more practical trainings on auditing methodologies. Finally, TCU has been promoting more active engagement by the academic community. There may be greater opportunities to tap into academic expertise, to better prepare future auditors and to establish agreements for the co-development of tools (surveys, research, apps, etc). Engaging the academic community can also bring different perspectives on cross-cutting governance issues.
1.5. Enhancing TCU’s monitoring and evaluation techniques to better assess its impact on governance
TCU could enhance its monitoring and evaluation techniques in order to generate, measure and communicate financial and non-financial impact of cross-cutting audit activities.
To encourage the monitoring and evaluation (M&E) of performance by the Public Administration,” TCU’s External Control plan (2015-2017) established an objective “to assess compliance with the decisions issued by TCU” (TCU, 2015b). TCU has a process for monitoring auditees’ compliance with TCU’s determinations and recommendations. For instance, when issuing ‘determinations’ in cases of non-compliance, TCU monitors the implementation according to an established deadline through analysis of documentation, or making inspections, which involves visiting the audited entity to collect data and information. In such cases, determinations are binding and there can be consequences for non-compliance. Unlike determinations, recommendations that focus on improvements to governance (i.e. efficiency effectiveness, economy) or transparency are not mandatory and are non-binding.
Communicating TCU’s M&E of its own work and results of its recommendations is largely posited in financial terms. TCU officials expressed the need to revisit the metrics used for calculating impact, given that relying on financial impacts has traditionally dominated discussions about TCU’s value-add. TCU is searching for ways to better quantify contributions to good governance through its audits and evaluations, recognising the results can be more qualitative and less tangible than other areas of work (e.g. judicial rulings and financial audits). Moreover, TCU has reduced in recent years the number of control activities, and subsequently the number of resulting recommendations. This has been an intentional effort of TCU, whose officials note that the current number of recommendations is unmanageable and does not facilitate easy tracking of the impact of audit recommendations.
To understand its impact and the results of its recommendations, particularly those meant to improve government performance, TCU could take actions to improve how it measures results. This could include efforts to more strategically asses and communicate qualitative, non-financial benefits that TCU provides to government and society. In addition, this could involve taking further development of ways for using M&E to generate, measure and communicate impact of cross-cutting audit activities. Specifically, TCU could consider the following:
-
TCU could further develop methods to generate and communicate a more aggregate measure of its value and benefits to society - both financial and non-financial; and
-
TCU could strengthen the coherence of its recommendations on governance-related issues, emphasising analyses of previous recommendations made to the Centre of Government.
TCU could further develop methods to generate and communicate a more aggregate measure of its financial and non-financial value and benefits to society.
To measure its own impact, TCU uses indicators and targets mapped out in the External Control Plan and Guidelines, and aggregates statistics based on individual activities. In 2015, TCU calculated its total potential benefit of its audit activities to be nearly 24 billion Brazilian Reais (BRL). TCU’s rate of return to the public was calculated at 13.4 BRL for every 1 BRL spent on TCU (TCU, 2015a). Such benefits are determined through the valuation of select activities, such as improvements in the economy, efficiency, efficacy or effectiveness of public entities (valued at approximately 235 million BRL in 2014), or the value of condemnations in 2015 (6.6 Billion BRL) (TCU, 2015a). These figures are made public in TCU’s annual report, along with others relevant to its performance and productivity, including the number of individuals convicted in debt or fraud, the value of condemnations, companies barred from public biddings and sentences pronounced.
While such indicators of TCU’s performance are beneficial for understanding the return on investment for taxpayers, TCU officials expressed concern that the discussion about TCU’s impact relies heavily on financial impacts. Indeed, TCU’s current processes for tracking recommendations and measuring a broader notion of impact faces many challenges, given the high volume of recommendations and the lack of a systematic process for measuring certain types of benefits. These benefits include the results of recommendations that focus more on governance issues and are elaborated on in performance audits. The benefits of SAIs can be generally categorised as follows:
-
Quantified financial benefits, such as TCU’s return on investment measure of financial benefits divided by financial costs. Only a handful of SAIs are in practice of calculating such a figure, such as Costa Rica, the United Kingdom and the United States.
-
Quantifiable non-financial benefits, such as the results that are deduced from perceptions surveys of auditees about the impact of SAIs’ work on their institution.
-
Qualitative benefits that cannot be quantified, which capture broader outcomes such as improving accountability or accuracy of performance indicators in government.
As part of nuancing its external control activities to longer-term impact on society, TCU could review its current monitoring approaches to assess its usefulness in providing a more aggregate picture of its overall benefits to society. Adopting a broader view of TCU’s impact, which includes quantifiable non-financial benefits and qualitative data, could offer a more realistic picture of TCU’s contributions to good governance and areas for improvement. TCU could expand beyond reporting on individual interventions, outcomes of its monitoring (monitoramento) and financial benefits, in order to better measure and communicate the value of governance related activities. Measuring and communicating benefits can be difficult and costly. It requires the capacity to make reliable, credible, complete, and accurate estimations. It also requires careful liaison with audited entities to ensure that benefits are not overstated. TCU can improve its chances of effective impact measurement and communication by considering its approach to this early in the audit planning process.
TCU could strengthen the coherence of its recommendations on governance-related issues, emphasising analyses of previous recommendations made to the Centre of Government.
ISSAI 12 stipulates that SAIs should enable those charged with public sector governance to discharge their responsibilities in responding to audit findings and recommendations and taking appropriate corrective action (principle 3) (INTOSAI, 2013a). The ability of an SAI to ensure take-up of its recommendations depends largely on the quality and clarity of recommendations. Equally important is the coherence between recommendations made to the same entity or on the same thematic topic or programme. Further, ISSAI 12 calls for SAIs to be a model organisation through leading by example. If TCU is to encourage coherence, communication and co-ordination in the Executive (Chapter 2), it too should be applying the same principles in the execution of its duties.
As mentioned, TCU has a system to track and manage recommendations. According to TCU officials, TCU could improve its mechanisms to ensure that recommendations are coherent with one another. The coherence has been challenged by the previously high number of recommendations. Since 2012, TCU has decreased the number of supervisory proceedings (fiscalizações) including audits and, as a result, reduced the number of associated non-binding recommendations to auditees (TCU, 2015a). This progression is welcomed by some TCU staff, who noted that even the current number of recommendations is unmanageable and does not facilitate easy tracking of their impact. As coherence between policies and programmes of government are important to avoid waste, duplication, fragmentation and overlap, so too is it important that SAIs are issuing messages to auditees that are, at least, non-conflicting.
One way TCU could promote greater coherence in its recommendations is to assess previous recommendations made to the Centre of Government (CoG), and then clarify any inconsistencies internally and externally. The CoG has a critical role to play in co-ordinating and overseeing the implementation of government-wide initiatives. As such, it is equally critical that TCU’s messages to the CoG, about the performance of the Brazilian government, are coherent and non-conflicting. Guidance to the CoG from TCU should be clear and support the CoG in discharging its critical functions. Such an analysis could discern the degrees of synergies, overlaps or contradiction between recommendations made in previous years. Enabling entities to discharge their responsibilities and take-up TCU’s recommendations relies on their clear understanding of those recommendations.
The analysis of coherence of recommendations could be replicated in particular thematic areas or for specific institutions, for particular laws, against which new recommendations could be checked in real-time. This would help to pacify TCU concerns, expressed by some officials, about the quality of outcomes and recommendations of governance-related work, which is a newer area of work relative to TCU’s other supervisory and accounts proceedings. For instance, an independent study of the implementation of the U.S. Government Accountability Office’s (GAO) recommendations analysed over 40,000 GAO recommendations from 1984 to 2014 using text analytics. Such a longitudinal analysis was facilitated by the GAO’s consistent and persistent approach to quantifying agency compliance. The study hypothesised that audits related to the GPRA and Modernization Act of 2010 (GPRAMA), which establishes a framework for taking a crosscutting and integrated approach to improve government performance, would result in recommendations that were more difficult for agencies to implement than others. The findings showed that recommendations related to GPRAMA had, in fact, a 2% higher success rate [in terms of agencies’ implementation] than recommendations unrelated to the GPRAMA (Deloitte, 2015).
References
Deloitte (2015), Accountability Quantified, Deloitte University Press, https://dupress.deloitte.com/dup-us-en/topics/analytics/text-analytics-and-gao-reports.html
Edelman (2016), Global Report, 2016 Edelman Trust Barometer, http://www.edelman.com/insights/intellectual-property/2016-edelman-trust-barometer/global-results/
European Court of Auditors (2015), “Work programme”, www.eca.europa.eu/Lists/ECADocuments/WP2015/WP2015_EN.pdf; INTOSAI (2004), “ISSAI 300: Fundamental Principles of Performance Auditing”, http://www.issai.org/data/files/33/45/70/E6/F93E7510D33F2E75CA5818A8/issai-300-english.pdf.
FFP (Fund for Peace) (2016), The Fragile State Index 2016, available at http://fsi.fundforpeace.org/, accessed 14 September 2016
INTOSAI (International Organisation of Supreme Audit Institutions) (2013a), ‘ISSAI 12: The Value and benefits of Supreme Audit Institutions – Making a Difference to the Lives of Citizens’, INTOSAI Professional Standards Committee, Copenhagen, Denmark, available at: http://www.issai.org/data/files/D8/53/3D/6F/79CD65107FA83C65BA5818A8/issai-12-e.pdf accessed 21 June 2017.
INTOSAI (2013b), ISSAI 200 – General Standards in Government Auditing, INTOSAI Professional Standards Committee, Copenhagen, Denmark, available at: http://www.issai.org/en_us/site-issai/issai-framework/3-fundamental-auditing-priciples.htm
Lonsdale, J. et al (2011), Performance Auditing: Contributing to Accountability in Democratic Government, Edward Elgar Publishing, Cheltenham, United Kingdom
National Audit Systems Network (2014) ‘Planning for Audits of Official Control Systems’ Version 1, February 2014, accessed October 2016, http://www.livsmedelsverket.se/globalassets/produktion-handel-kontroll/vagledningar-kontrollhandbocker/vagledningar-och-information-fran-eukommissionen/risk-based-planning-for-audits-of-official-control-systems-february-2014
OECD (2016a), Supreme Audit Institution and Good Governance: Oversight, Insight and Foresight, OECD Public Governance Reviews, OECD Publishing, Paris. https://doi.org/10.1787/9789264263871-en
OECD (2016b), ‘Navigating Wicked Problems’, OECD Insights Blog, http://oecdinsights.org/2016/10/04/navigating-wicked-problems/
OECD (2016c), Revenue Statistics in Latin America and the Caribbean 2016, OECD Publishing, Paris. https://doi.org/10.1787/rev_lat_car-2016-en-fr
OECD (2016d), «Better Life Index 2015», OECD Social and Welfare Statistics (database). https://doi.org/10.1787/data-00823-en as cited in OECD (2015c), OECD Economic Surveys: Brazil 2015, OECD Publishing, Paris. https://doi.org/10.1787/eco_surveys-bra-2015-en
OECD (2015a), Water Resources Governance in Brazil, OECD Studies on Water, OECD Publishing, Paris. https://doi.org/10.1787/9789264238121-en
OECD (2015b), “Citizen satisfaction with public services”, in Government at a Glance 2015, OECD Publishing, Paris. https://doi.org/10.1787/gov_glance-2015-56-en, with data available at: http://stats.oecd.org//Index.aspx?QueryId=66864
OECD (2015c), ‘Education Policy Outlook: Brazil’ OECD Publishing, Paris. http://www.oecd.org/edu/Brazil-country-profile.pdf
OECD (2014a) ‘Centre Stage: Driving Better Policies from the Centre of Government’ OECD Publishing, Paris. https://www.oecd.org/gov/Centre-Stage-Report.pdf
OECD (2014b), Chile’s Supreme Audit Institution: Enhancing Strategic Agility and Public Trust, OECD Public Governance Reviews, OECD Publishing, Paris. https://doi.org/10.1787/9789264207561-en
Rigsrevisionen (Danish SAI) (2016), “The strategic analysis”, 17 August, http://uk.rigsrevisionen.dk/how-we-audit/annual-audit/strategic-analysis/
Sabogal, David, Helen Bellfield and Simone Bauch, “Amazonia Security Agenda: Assessing policy coherence in Brazil, Colombia and Peru, the Water-Energy-Food Nexus approach”, Policy brief, the Climate and Development Knowledge Network (CDKN), accessed online, September 2016, http://globalcanopy.org/sites/default/files/documents/resources/Regional%20Policy%20Brief%20English%20FINAL.pdf
Santiso, C. (2009), Political Economy of Government Auditing, Taylor and Francis Publishing, Abington UK, http://www.routledge.com/books/details/9780415477734/.
Social Progress Imperative, accessed online 20 February 2017, http://www.socialprogressimperative.org/
TCU (Tribunal de Contas da União) (2017), PORTARIA-SEGECEX Nº 5 DE 7 DE MARÇO DE 2017 [Department of Institutional Relations for Control on Fight Against Fraud and Corruption], Brasilia.
TCU (2016a), Framework for Evaluation of the Centre of Government, TCU Publishing, Brasilia, http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A25454C5A801545DC1433145ED
TCU (2016b), ‘Gestão de pessoas’ http://portal.tcu.gov.br/transparencia/gestao-de-pessoas/, accessed Oct 2016.
TCU (2016c), Trimestral activity report – Trimester 1 (Relatório Trimestral 2016 - 1º trimestre), http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A1550E2AD30155129EACD506FE
TCU (2016d), Trimestral activity report – Trimester 2 (Relatório Trimestral 2016 - 2º trimestre), http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A256ED46890156FC6A8C537FC0
TCU (2016e), General Governance Index, accessed 20 February 2017, http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A14F613FB5014F6B5206735279
TCU (2015a), 2015 Annual Report (Relatório Anual - 2015), http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A153D422DA0153E2950D8064E1
TCU (2015b), Strategic Plan of Brazil’s Federal Court of Accounts, 2015-2021 (Plano Estratégico do Tribunal de Contas da União para o período 2015-2021) PORTARIA-TCU Nº 141, DE 1º DE ABRIL DE 2015, Brasilia, http://portal.tcu.gov.br/tcu/paginas/planejamento/2021/index.html.
TCU (2015c), TCU Resolution 269/2015, ACÓRDÃO TCU 269/2015, http://www.lexml.gov.br/urn/urn:lex:br:tribunal.contas.uniao;plenario:acordao:2015-02-11;269, accessed 21 June 2017.
TCU (2015d), Report of the Secretary General of External Control (Relatório da Secretaria-Geral de Controle Externo), Brasilia, accessed July 2017. http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A2572667DE0157298FA4C94D16&inline=1
TCU (2015e), External Control Plan of Brazil’s Federal Court of Accounts (Plano de Contrôle Externo do Tribunal de Contas da União) April, 2015, Brasilia, http://portal.tcu.gov.br/lumis/portal/file/fileDownload.jsp?fileId=8A8182A153234E0A01535D1006D60567.
TCU (2014a) Annual activities report (Relatório de Atividades 2014), Brasilia, http://portal.tcu.gov.br/transparencia/relatorios/relatorios-de-atividades/, accessed 20 February 2017
TCU (2010) Performance Audit Manual, TCU: Brazilian Court of Audit, Brasilia, http://psc-intosai.org/data/files/21/60/D0/A0/81B07510C0EA0E65CA5818A8/tcu_performance_audit_manual.pdf
TCU (2007), ‘Supreme Audit Institutions and Financial Sustainability in the Modern State’ presented at EUROSAI/OLACEFS Lisbon, 2007, http://www.tcontas.pt/eventos/EurosaiOlacefs/Docs/3ST/Brasil-TCU/WS3BR-engl.pdf, Accessed 30 Sep 2016
United Nations (2014), General Assembly resolution 69/228, Promoting and fostering the efficiency, accountability, effectiveness and transparency of public administration by strengthening supreme audit institutions, A/RES/69/228 (19 December 2014), available at http://www.un.org/en/ga/search/view_doc.asp?symbol=A/RES/69/228
World Bank (2014), Worldwide Governance Indicators 2014, http://info.worldbank.org/governance/wgi/index.aspx
World Bank, World Governance Indicators Project, http://info.worldbank.org/governance/wgi/index.aspx#home;
World Happiness Report, accessed online, 20 February 2017, http://worldhappiness.report/
Notes
← 1. This definition does not cover other units, offices, and commissions (e.g. offices for sport or culture) that may report directly to the President or Prime Minister but are, effectively, carrying out line functions that could be carried out equally well by line ministries. COG can include ministries of finance and treasuries, and government-wide planning and management ministries. TCU’s definition of Centre of Government is the: Administrative structure of central institutions serving the Executive, representing the strategic core of the federal government for strategic planning, policy and technical co-ordination of government actions, monitoring the performance and communication of decisions and government achievements. It consists of state actors with great potential interference in the public policy cycle, which play cross central and government functions, even if they are not within the Chief Executive’s Office and does not serve him only (TCU, 2016a).
← 2. Wicked problems are policy issues that are dynamic, poorly structured, persistent and social in nature, and highly intertwined with other social issues, such as climate change, immigration, poverty, nutrition, education, or homelessness (OECD, 2016a).
← 3. This expenditure is for all levels of education combined.
← 4. OECD’s Better Life Index measures life satisfaction by how people evaluate their life as a whole rather than their current feelings. General satisfaction with life is rated on a scale from 0 to 10.
← 5. The number of activities requested by Congress in quarter 1 of 2016 was 19.1%, whereas 80.9% were undertaken at TCU’s discretion. The number of Congressional requests was much higher in quarter 2 at 48.8% (51.2% at TCU’s initiative). This substantial increase marks the highest proportion of supervisory proceedings (fiscalizações) resulting from Congressional requests in the last 2 years in any given quarter. Among the control activities completed in the first semester of 2016, 33.25% were requested by Congress and the remaining 65.75% were at the Court’s own initiative. This information is summarised from TCU annual activity reports, that can be found online (TCU, 2016c, d; 2015a, 2014a).