Chapter 3. Taking a risk-based approach to strengthen integrity and mainstream internal control in the Colombian public governance system

This chapter highlights the added value of a robust, contemporary internal control system in mitigating the challenges of allocating scarce resources and delivering impact for Colombian citizens. Risk management is the driving force behind governments’ efforts to deal with uncertainties and threats, including fraud and corruption risks, which hamper the achievement of the strategic objectives of public organisations. The internal audit’s role is to provide reasonable assurance that management systems and defence arrangements are properly set to address low performance, wrongdoing, fraud and corruption.

A solid internal control framework is the cornerstone of an organisation’s defence against corruption, and consists of processes, policies, devices, practices or other activities which act to minimise negative risks or enhance positive opportunities. This can include all tangible and intangible factors that enable an organisation to identify and appropriately respond to both internal and external uncertainties, whether these are operational, financial, or compliance-related. An effective internal control framework should ultimately help the organisation comply with its mandate and any relevant legislation, safeguard an organisation’s assets, and facilitate internal and external reporting.

While it is senior managers who are primarily responsible for implementing internal controls and monitoring their effectiveness, all officials in a public organisation – from the most senior to junior staff- – have a role to play in identifying risks, deficiencies and ensuring that internal controls address and mitigate these in a cost-effective manner. Indeed, every staff member should be encouraged to continuously contribute to the development of better systems and procedures that will enhance integrity and improve the organisation’s resistance to corruption.

Internal audit is the next pillar of defence against corruption and provides objective assurance that risk management and internal controls are functioning properly. A proper internal audit monitoring and assurance function ensures that internal control deficiencies are identified and communicated in a timely manner to those actors responsible for taking corrective action. The monitoring process more concretely involves establishing a solid approach for designing and executing monitoring procedures that are prioritised based on risk, assessing and reporting the results, including following up on corrective action where necessary.

It is important to note that, while risk, control and audit functions are essential in the fight against corruption, they are also necessary ingredients for greater accountability, better management and cost-effectiveness. To this end, controls help organisations run more smoothly, reduce costs, and avoid waste. They also help hold public officials to account for their actions and to report to the public and oversight institutions on performance and value-for-money achieved.

Colombia’s Administrative Department of the Public Service (Departamento Administrativo de la Función Pública, or DAFP) is the entity responsible for developing and overseeing policies, standards and tools on internal control, including the risk management and internal audit functions.

This chapter examines the maturity and integration of internal control processes and activities. It also looks at the assignment of roles and duties in Colombia’s public administration in respect to these functions along the three lines of defence model. Thirdly, it considers the extent to which they are based on the principles of risk management, balanced and cost-effective controls, and effective assurance oversight.

Embedding fraud and corruption risk management in Colombian public organisations

The DAFP should further engage in a risk-based approach as the bedrock to consolidate a control environment that is non-conducive to fraud and corruption

In many OECD member and partner countries, fraud and corruption control is designed around reactive measures since the real action starts once an incident is discovered following which an investigation is conducted and appropriate disciplinary or other action is taken against employees and external parties involved. In those entities, little or no emphasis is placed on the need for introducing proactive fraud and corruption risk management mechanisms. Proactive fraud and corruption risk management aims to support public organisations in achieving their mission and strategic goals by ensuring that taxpayers’ money and government entities serve their intended purposes. A properly conducted risk assessment is the bedrock of understanding the vulnerabilities of a system and put in place the right (i.e. proportionate and effective) controls to deal both with inherent but mainly with residual fraud and corruption risks. A solid fraud and corruption control strategy should detail the entity’s intended actions in implementing and monitoring the entity’s fraud and corruption prevention, detection and response initiatives.

The European Anti-Fraud Office (OLAF) is clearly highlighting the value of all public organisations to engage into concrete actions to proactively address fraud and corruption risks (Box 3.1).

Box 3.1. The European Commission’s Anti-Fraud approach

The European Anti-Fraud Office (Office Europeéen de Lutte Antifraude; OLAF) recommends that public organisations adopt a proactive, structured and targeted approach to managing the risk of fraud and corruption. For all organisations using public funding, the objective should be proactive and proportionate anti-corruption measures with cost-effective means.

All public entities should be committed to zero tolerance to fraud, starting with the adoption of the right tone from the top. A well-targeted fraud risk assessment, combined with a clearly communicated commitment to combat fraud can send a clear message to potential fraudsters. It should be noted that effectively implemented robust control systems can considerably reduce the fraud risk but cannot completely eliminate the risk of fraud occurring or remaining undetected. This is why the systems also have to ensure that procedures are in place to detect frauds and to take appropriate measures once a suspected case of fraud is detected.

A dedicated fraud and corruption risk management framework can provide a holistic approach on how public organisations can set up their defences to effectively prevent, detect and respond to fraud and corruption acts.

Source: http://ec.europa.eu/regional_policy/en/information/publications/guidelines/2014/fraud-risk-assessment-and-effective-and-proportionate-anti-fraud-measures

In Colombia, anti-corruption risk management became obligatory for all public entities in 2011 with Law 1474, the Anti-corruption Statute. Corruption risk identification and assessment started as an add-on exercise in 2012, promoted by the Transparency Secretariat (Secretaría de Transparencia, or ST). From the beginning, the methodology was widely based on the existing internal control model (i.e. Modelo Estándar de Control Interno, or MECI). Based on the experiences after engaging in this exercise, a second version of the methodology has been issued in 2015 that aligns even better and more explicitly with the MECI as the latter was revised after the introduction of the COSO 2013 Internal Control-Integrated Framework.

Having two separate exercises for mapping and assessing on the one hand institutional-business risks and on the other hand corruption risks comes along with both strong and weak points. Currently in Colombia, although the two methodological approaches are quite aligned and based on the same principles, many institutions stated that they are developing two different sets of risk registers and maps for each one of these risk categories. This is also due to the different reporting channel requiring that the corruption risk maps be sent to the Transparency Secretariat. Unavoidably, this comes along with potential duplications, overlaps and waste of resources depending on how these two exercises are conducted in practice.

In turn, having a dedicated fraud and corruption risk assessment procedure signals the importance, across government, of effectively managing these risks at the entity level. Heads of institutions and other politically appointed personnel as well as civil servants become aware of the importance of this exercise for strengthening integrity and accountability arrangements. This does not mean that the corruption risk management competes in any way with the institutional risk management function. They are both important elements of the public governance systems and the ultimate goal is to integrate these processes in day-to-day operations. According to DAFP, in practice, entities are advised that the institutional risk map must be one, and must integrate both corruption and operational risks, which may affect the achievement of the entities’ objectives.

Therefore, to benefit from the synergies and further align both exercises while reducing transaction costs, Colombia could consider gradually integrating even more corruption risk management into the MECI processes and activities.

The following figure from the COSO/ACFE’s Fraud Risk Management Guide clearly links the 17 principles of the COSO 2013 Internal Control-Integrated framework, which despite the differences are to a large extent reflected in the last version of the MECI, with the five (5) anti-fraud principles of this guidebook (Figure 3.1). This approach illustrates the close relationship between the internal control arrangements and an effective fraud and corruption risk management strategy.

The Planning Unit in each public entity is responsible for supervising both the MECI and the corruption risk-mapping process. Interviews with Heads of the internal control offices (Oficinas de Control Interno, or OCI) from different public organisations indicated a solid understanding of the need to directly involve the line managers and staff, i.e. the risk owners, in the risk assessments and relevant activities. OCIs, on the other hand, are responsible for providing assurance over the quality of the whole process and ensuring that risks are identified, assessed and properly mitigated as part of the functions of the first and second lines of defence.

The institutional and operational arrangements concerning corruption risk management in entities that are regulated and supervised by the Financial Superintendencia (Superintendencia Financiera de Colombia), like the National Fund for Development Projects (Fondo Financiero de Proyectos de Desarrollo, FONADE), seem to be very well structured and implemented. Indeed, these entities are subject to stricter rules, in general have more advanced management systems, and have more advanced internal control and risk management processes because of corporate governance requirements and relevant regulations and international good practices (such as the OECD/G20 corporate governance guidelines). It is important to note that FONADE’s fraud and corruption risk assessment (SARFC) is part of the broader enterprise risk management system which also involves other risk categories like operational risks (SARO) and money laundering and financing terrorism risks (SARLAFT). In addition to the Planning Unit which is responsible for risk management, as it is the case in other Colombian public organisations, in FONADE we also encounter a dedicated Risk Committee.

The integrity attributes of the internal control environment need to be strengthened and the right tone at the top demonstrated to create the necessary preconditions for effectively managing fraud and corruption risks

As mentioned above, managers are primarily responsible for designing and establishing a solid control environment demonstrating the entity’s commitment to ethical values. However, when it comes to implementing concrete integrity attributes and tools (e.g. code of ethics requirements, disclosing conflict-of-interest situations, ethical dilemma trainings etc.) everyone has a role, from senior and middle management to the staff. The organisation’s ethical values, and the processes and procedures underpinning those values have to be communicated, applied and, if necessary, enforced throughout the organisation. This is not an easy task. Human perceptions and behaviour can influence the actual implementation of policies and ethical codes.

As emphasised in Chapter 2, for public servants to understand the difference between getting a copy of the code when entering the organisation and actually contributing to a sustainable and functional ethical control environment, raising awareness and training activities across the entity over integrity requirements and ethical standards should be an absolute priority for Colombia. Furthermore, management needs to measure, monitor, and in case of undesired conduct by staff, act and communicate its prompt reaction. Managing the control environment should not be done ad hoc but should be part of planning, daily operations, and standard evaluation and monitoring processes. Internal auditors, as “key agents of change” in the organisation, should assess the control environment as part of their assurance mandate, and motivate management to address flaws and inefficiencies regarding the effectiveness and the maturity of the control environment. Box 3.2 depicts concrete measures that Colombia could adopt or strengthen, if already applied, towards an optimal anti-corruption control environment.

Box 3.2. Key measures towards developing a non-conducive to corruption environment

All management plans, regardless of level, should reflect the organisation’s values and ethics
Requiring an individual “ethical contract’” or code of conduct to be signed between recruiter and recruit at the moment of first entry into service and periodical (e.g. annual) re-signing
Dilemma training during which the organisation’s values are explained in very concrete situations (for all levels of the organisation, including management)
Workshops on ethics and values including some especially for senior and middle management
HR procedures for hiring, evaluation and dismissal must reflect and openly support the organisation’s mission and values
The organisation’s values are included in function profiles and job descriptions
Ethical clauses in procurement processes and in contracts with external suppliers
Ethics co-ordinators with specific responsibilities to promote and enhance awareness of ethics
The key values of the organisation are publicly displayed
Developing a process to report suspected violations of the organisation’s code of conduct

Source: Public Internal Control Systems in the European Union, Position Paper 2015

In addition to the recommendations in Chapter 2, Colombia could also ensure, through further sensitising and training, that politically-appointed personnel, public managers, civil servants and especially public servants working in planning units and OCIs recognise corruption and integrity violations as a threat to the internal control environment.

Efforts to further tailor and mainstream awareness-raising and training activities could include the following:

To move towards further aligning the corruption and integrity risk assessment with the MECI risk assessment procedure, the methodology for corruption risk mapping could become part of standard MECI capacity building and training seminars.
Building on past experiences, specific modules on preventing, detecting and responding to corrupted practices could be additionally developed in collaboration with Academia and the Higher School of Public Administration (Escuela Superior de Administración Pública, or ESAP). This should be tailored to the needs of planning units personnel, staff of OCIs as well as senior public managers, especially those in high-risk areas like human resource and financial management as well as public procurement.
These modules should emphasise the wide variety of corrupt practices and types of integrity violations to ensure that these are identified as risks by public officials and that adequate and cost-effective controls are put in place.
The Colombian public administration policies should emphasise the value of taking up role model and the tone-at the top for promoting ethics. Concrete actions could include:
1. Screening managers on traits favouring ethical behaviour and testing ethical compliance during management selection procedures;
2. Seminars and awareness campaigns on ethics and values for management both collectively and individually;
3. Self-assessment tools for managers (evaluation questionnaire) including ethical aspects;
4. 360° evaluations for senior managers as well as managers in high risk positions (with evaluations including ethical aspects)

The concrete role and responsibilities of Internal Control Offices in preventing, detecting and responding to fraud and corruption schemes need to be better defined

The leading fraud and corruption risk management models among OECD member and partner states underscore that the primary responsibility for preventing and detecting corruption rests with the management and the staff of public organisations (first and second line of defence functions), as well as with enforcement agencies such as police and anti-corruption institutions. Internal audit units should not prioritise on tackling fraud and corruption but rather on fostering an environment that is not conducive for this kind of schemes. Moreover, internal auditors should not be directly involved in fraud and corruption investigations.

The OCIs in Colombia undertake the role of a contemporary internal audit function as an assurance provider residing in the third line of defence within the overall institutional internal control system. By contributing advice and insight into an organisation’s overall governance framework, internal auditing plays a vital role in being an agent of positive change in an organisation and prevents the occurrence of fraud and corruption. Therefore, the OCIs can play a strong, value-oriented, and objective role in corruption awareness and prevention, although they should not be considered as the primary responsible actors. In this framework, the role of the OCI’s staff in corruption risk management should be carefully defined to avoid duplications and gaps in the control arrangements.

The role of OCIs with respect to investigating cases of fraud or corruption should also be clearly defined in the internal audit charter. In any case that internal audit accepts some form of responsibility for risk management in these areas, this should be defined in the charter making clear that the work is not carried out as part of the internal audit role and identifying how internal audit independence and objectivity is safeguarded. In Colombia, it seems that in some cases politically-appointed personnel and senior public managers expect the internal audit to be more actively involved in managing fraud and corruption risks.

The international internal audit standards (IIA, International Professional Practices Standards) recognise the importance of fraud-risk management and that internal auditors have an important role to play. However, this role focuses on providing assurance. The internal audit role is not to manage fraud risks on behalf of the organisation, but to provide an assurance that all risks, including fraud and corruption risks, are being managed effectively. If, during an audit assignment, staff of the OCI identify control weaknesses that could allow fraud, or find evidence that fraud has been, or is being, perpetrated, they should be able to refer to the relevant internal audit procedures on handling suspected fraud and consult the organisation’s fraud response plan, which has to be put in place. This will normally identify what they need to do and whom they need to alert.

In this framework, the heads of OCIs may need to extend the audit work and design additional tests directed towards identifying activities which may be indicators of fraud and carefully examine the available evidence in order to decide whether there is clear evidence of fraud to recommend an investigation. A key consideration would be at what point to alert management (e.g. staff with designated anti-fraud responsibilities or a Money Laundering Reporting Officer). This will be an important decision for the Head of the OCI based upon the individual circumstances and the formally established policies, procedures and responsibilities in the respective public organisation.

The Institute of Internal Auditors’ International Professional Practice Framework (IPPF) defines fraud as “any illegal act characterized by deceit, concealment, or violation of trust (…) perpetrated by parties and organisations to obtain money, property, or services; to avoid payment or loss of services; or to secure personal or business advantage”. The Institute of Internal Auditors’ Standard 2110 on Governance (IPPF 2015) specifically refers to the responsibility of internal audit to evaluate the existing situation and submit appropriate recommendations to improve the governance in order to promote the right ethical values and principles inside the entity. Furthermore, there is a practical guide on Evaluating Ethics-related Programs and Activities (IIA 2012).

The following principles relate to the role of Internal Audit in responding to fraud and corruption risks (IIA’s IPPF):

1210.A2 (Proficiency): “Internal auditors must have sufficient knowledge to evaluate the risk of fraud and the manner in which it is managed by the organisation, but are not expected to have the expertise of persons whose primary responsibility is detecting and investigating fraud”.
1220.A1 (Professionalism): “Internal auditors must exercise due professional care by considering the Probability of …, fraud, or … ”
2060: “Chief Audit Executives (CAE) must report periodically to senior management and the board … on fraud risks … ”
2120.A2 (Risk Assessment): “The internal audit activity must evaluate the potential for the occurrence of fraud and how the organisation manages fraud risk”;
2210.A2 (Engagement Objectives): “Internal auditors must consider the probability of significant errors, fraud, noncompliance, and other exposures when developing the engagement objectives”.

Colombia should therefore ensure that the internal audit charter clearly defines the procedure for communicating corruption cases and financial violations to the competent authorities internally and externally, i.e. the disciplinary internal control, the Comptroller General (Contraloría General de la República, or CGR), the Inspector General (Procuraduría General de la Nación, or PGN), and the General Prosecutor (Fiscalía). This should be done within a limited time period from the date of case detection or event occurrence.

Beyond these reporting procedures, internal audit should focus on a proactive approach through risk-based internal audit techniques by detecting red flags and symptoms of corruption. Audit tools and techniques to detect corrupt activities can include the evaluation of internal controls such as administrative controls and accounting; data mining with the use of Computer-Assisted Audit Techniques (CAATs); data gathering tools such as interviewing, observations questionnaires, checklists and sampling; and analytical reviews such as ratio analysis and variance analysis.

While conducting audit missions, the auditors should act to identify fraud and corruption indicators that can be recognised in most of the core business processes. To be successful in recognising these indicators, auditors must rely on their technical experience, professional judgment and good understanding of how potential fraud and corruption acts can be committed. Audit strategies should target areas and operations prone to fraud and corruption by developing effective risk indicators (red flags). To enhance auditing skills and capacity for fraud detection, Colombia could consider developing “Fraud Auditing Guidelines” to standardise and mainstream anti-fraud processes and equip internal auditors with methodological standards and tools. The following box provides a quick overview of the United Kingdom’s approach to the tasks of internal audit in relation with fraud and corruption (Box 3.3).

Box 3.3. Role of internal audit in fraud and corruption

It is not a primary role of internal audit to detect fraud and corruption. Internal audit’s role is to provide an independent opinion based on an objective assessment of the framework of governance, risk management and control. In doing so, internal auditors may:

Review the organisation’s risk assessment seeking evidence on which to base an opinion that fraud and corruption risks have been properly identified and responded to appropriately (i.e. within the risk appetite). • Provide an independent opinion on the effectiveness of prevention and detection processes put in place to reduce the risk of fraud and/or corruption.
Review new programmes and policies (and changes in existing policies and programmes) seeking evidence that the risk of fraud and corruption had been considered where appropriate and providing an opinion on the likely effectiveness of controls designed to reduce the risk.
Consider the potential for fraud and corruption in every audit assignment and identify indicators that crime might have been committed or control weaknesses that might indicate a vulnerability to fraud or corruption.
Review areas where major fraud or corruption has occurred to identify any system weaknesses that were exploited or controls that did not function properly and make recommendations about strengthening internal controls where appropriate.
Assist with, or carry out investigations on management’s behalf. Internal auditors should only investigate suspicious or actual cases of fraud or corruption if they have the appropriate expertise and understanding of relevant laws to allow them to undertake this work effectively. If investigation work is undertaken, management should be made aware that the internal auditor is acting outside of the core internal audit remit and of the likely impact on the audit plan.
Provide an opinion on the likely effectiveness of the organisation’s fraud and corruption risk strategy (e.g. policies, response plans, whistleblowing policy, codes of conduct) and if these have been communicated effectively across the organisation. Management has primary responsibility for ensuring that an appropriate strategy is in place and the role of internal audit is to review the effectiveness of the strategy.

Source: United Kingdom, HM Treasury (2012), Fraud and the Government Internal Auditor, January 2012.

Some concrete proposals for framing the role of internal audit in managing fraud and corruption risks could include the following actions:

While conducting audit missions, OCIs staff becomes aware of typical corruption scenarios and red flags that may be inherent to individual processes. This information can be used to develop training material for managers and staff in the respective operations area.
Internal auditors can serve as expert content providers for online training development (e.g. code of conduct, conflict of interest, harassment etc.) and corruption education websites providing information to process owners on types of corruption and the root causes for potential corruption acts in a process. This site can provide information about the impact of poor segregation of duties and typical “red flags” for low efficiency controls within particular business processes.
Internal auditors can also support process reengineering and development by providing insights on control points in the process that may present opportunities for corruption, if not properly managed.
Internal audit may perform a level of fraud risk assessment in the framework of developing its annual audit planning. Although not responsible for entity wide risk assessment, OCIs personnel can act as impartial experts and facilitate meaningful discussions between different business areas to mitigate the “silo” approach effect and vet out corruption risks

The use of data analytics and big data could be further explored and leveraged to strengthen transparency and support a pre-emptive risk-based approach to tackle fraud and corruption

The risk management exercise heavily depends on the capacity and the knowledge of the staff involved as well as the quality of the data and input used in each one of the relevant activities, i.e. risk identification and assessment, evaluation of the effectiveness of existing controls, identifying corruption patterns and historical trends. As operational, governance and control data become more readily available, internal auditors, risk officers and line managers should work together to identify data streams that can be monitored and analysed for uncertainties and anomalies.

Forensic Data Analytics (FDA) can be a valuable ally in preventing and detecting fraud and corruption by leveraging the available information in government data assets. It enables identification of meaningful patterns and correlations in existing historic data to predict future events and assess the reasons for various fraudulent activities. Advanced or less mature FDA tools are currently in use in several public organisations with positive impact in curbing corruption. One of the biggest challenges lies with the fact that many of the government data are rather unstructured and thus difficult to be combined and assessed. In most public organisations, data exists in structured (simple or more sophisticated databases) and unstructured forms (e-mails, word processing documents, multimedia, video, PDF files, spreadsheets, social media). Forensic unstructured analytics techniques can turn raw data into formats that can be used to generate evidence-based information to prevent or detect fraudulent and corrupted practices. The 2014 Association of Fraud Examiners (ACFE) report identifies the proactive data monitoring/analysis as one of the most effective tools for anti-corruption control, reducing losses due to corruption and the duration of corruption schemes. Moreover the 2016 ACFE report highlighted that 36.7% of victim organisations that were using proactive data monitoring and analysis techniques as part of their anti-fraud program suffered fraud losses that were 54% lower and detected the frauds in half the time compared to organisations that did not use this technique.

The following box illustrates a concrete application of the added value of using data analytics to tackle health sector corruption (Box 3.4).

Box 3.4. Detecting health corruption through «fraud audit» in Calabria, Italy

In Calabria, countless investigations in healthcare have corruption as both a crime and a conspiracy, including mafia infiltration. In his report to the Italian Parliament on 27 February 2009, Renato Brunetta, then Minister of Public Administration and Innovation, showed that Calabria was in first place for corruption in healthcare. Still, much corruption remains hidden; despite the Laws on Checks and Controls, healthcare organisations previously lacked a comprehensive system of control of both administrative and economic performance.

Corruption in public administration is a very complex problem from many facets. In general, the employee, the manager or the general manager of a public body which deliberately violates the laws to reap illicit proceeds from the management of public funds, does not act alone. Corruption is based on the system of so-called complex networks at multiple levels. In order to unearth illegal activities and permit action to be taken, the Business Information Service (BIS) devised a methodology, implemented by the Provincial Health Authority of Catanzaro, which uses data management to locate administrative and accounting fraud in health companies. With a budget of around EUR 12 000 a year and 8 staff, the ‘fraud audit’ of Catanzaro employs internal controls and a set of IT-centred procedures and techniques to programme and subsequently monitor business operations in order to find clues to the possible mechanisms of corruption, in three areas:

First, systematic analysis was made of accounting documents and supplier invoices to discover double-billing, invoices not due, and higher-than-contractual invoiced amounts. Special software developed by the BIS was employed to apply Benford’s Law (which compares the frequency with which numbers actually appear with expected patterns) to analyse the distribution of all the figures related to invoice number, date and amounts for each health company. Risk of corruption was identified in 0.1% of the 12 000 documents checked. Follow-up found invoices for two companies with the same number but different dates, an invoice for purchasing disposable razors with the purchase order priced at EUR 9.00 per piece rather than the contracted cost of EUR 0.084, and increases in the cost of supply for chemical analysis slides and electrolyte solution between award and supply of 50 times and 100 times respectively.

Second, tenders for the supply of goods and services were evaluated where the number of participating companies was less than three, to discover contract awards at risk of illegitimacy. In ten cases, tender
awards had been made with the participation of a single company; in those cases where an offence was revealed, the matter was remitted to the competent authorities.
Finally, monitoring of violations of the computer network through a special “sniffer” programme uncovered data theft and hacking by both internal and external sources. This exposed two healthcare services company that were bypassing the system of firewalls and proxies, and which was referred to the police for investigation.

Source: European Commission (2015), Quality of Public Administration-A Toolbox for Practitioners, April 2015

There is strong evidence that public organisations in OECD member states benefit from introducing data analytics tools and techniques to enhance their ability to (IIA’s Global Technology Audit Guide):

Identify internal control system weaknesses;
Examine 100% of transactions compared to sampling;
Compare data from different applications;
Perform tests designed for fraud detection and control verification;
Automate tests in high-risk areas and
Maintain logs of analytics performed.

Data analytics can support Colombia in its effort to ensure that corruption risk management is not limited to a stand-alone exercise. Data mining and data matching techniques will allow Colombian public organisations to fully identify all potential risks and to use structured and unstructured data to better understand the potential impact of a range of risks. By embedding data analytics into the risk governance function, Colombian administration could monitor performance through risk sensitivity analysis, model key risk events scenarios, and become more risk intelligent in developing intervention and mitigation strategies. The following box depicts the basic steps of using data analysis of predefined fraud tests to facilitate continuous monitoring and auditing techniques (Box 3.5).

Box 3.5. Using data analytics to tackle fraud and corruption

Especially in the area of tackling corruption, the process involves gathering and storing relevant data and mining it for patterns, discrepancies, and anomalies. In the era of digitisation and e-government almost every single corrupted act leaves behind a trail of digital fingerprints. Leading public organisations in corruption prevention are taking advantage of new tools and technologies to harness their data to sniff out instances of corruption, ideally before they fully unfold. Data analytics can enhance traditional rule-based methods to detect wrongdoing and also provide the evidence to assess performance of existing controls for constant improvement since potential perpetrators and corruption schemes are unrelenting and constantly evolving. Using data analytics enable to find root issues, identify trends, and provide detailed results. Figure 3.2 illustrates the different maturity levels of data analytics’ contribution to design evidence-based fraud and corruption risk mitigation strategies.

Data analytics techniques are not strictly associated with complicated and expensive infrastructure and structured data assets. The following box provides a brief overview of Benford’s law which has been proven a valuable tool to detect fraudulent and corrupted behaviour. Benford’s law can be applied by using a simple excel sheets, which is commonly used by internal auditors in their daily operations to identify unusual data patterns that may signal the presence of fraud and corruption (Box 3.6).

Box 3.6. Benford’s law can be a cost-effective way to identify fraud

Benford’s law, also called “the first-digit law,” was made famous in 1938 by Physicist Frank Benford, who after observing sets of naturally occurring numbers, discovered a surprising pattern in the occurrence frequency of the digits one through nine as the first number in a list. In essence, the law states that in numbered lists providing real-life data (e.g., a journal of cash disbursements and receipts, contract payments, or credit card charges), the leading digit is one almost 33 percent (i.e., one third) of the time. On the other hand, larger numbers occur as the leading digit with less frequency as they grow in magnitude to the point that nine is the first digit less than five percent of the time.

Benford’s law allows to test certain points and numbers and to identify those ones that appear more frequently than they are supposed to and therefore they are suspect. The practical applications of this theory have been the downfall of fraudsters and a boon to fraud examiners

For example, let us assume an employee is committing fraud by creating and sending payments to a fictitious vendor. Since the amounts of these fraudulent payments are made up rather than occurring naturally, the leading digit of all fictitious and valid transactions will no longer follow Benford’s law. Furthermore, assume many of these fraudulent payments have three as the leading digit, such as $39, $322 or $3 187. By performing a first-digit test on the disbursement data using Benford’s law, auditors should see the amounts that have the leading digit three occur more frequently than the usual occurrence pattern of 12.5%.

Source: Global Technology Audit Guide (GTAG), IPPF – Practical Guidance, Data Analysis Technologies, Institute of Internal Auditors, 2011 and http://www.theiia.org/intAuditor/itaudit/archives/2008/february/puttingbenfords-

To promote the use of data analytics techniques and tools, Colombia could therefore establish a working group to assess capability and technology, and identify a pilot organization to realise measurable “quick wins”. Furthermore, conducting consultation workshops with OCIs heads and staff to map down the current state of data analytics and identify opportunities and concrete steps to move forward could build awareness and support from auditors, management and staff dealing with fraud and corruption risks. The pilot should test implementation tools and infrastructure and provide evidence of quantified impact and benefits. The roll-out phase should include additional pilots and concrete steps to incorporate data analytics into corruption risk assessment and audit planning. In this action plan some concrete activities could include.:

Heads of the OCIs must be in the forefront of demonstrating the added value of investing in effective data analysis tools and illustrating how the data analytics will support the fight against corruption.
OCIs can start using simple data analytics tools and focus on areas where the benefits are clear and data already available
Assigning to the OCIs staff with even basic knowledge of data analysis supported by tailored training activities can have tangible results since with expertise, significant analysis can be done in readily available tools, such as Microsoft Excel.

Integrating internal control for good and accountable public governance

DAFP should further work on bridging the implementation gap and mainstreaming internal control functions in the management systems of public organisations

Many OECD member and partner states are still facing serious challenges with bridging the implementation gap between their conceptual internal control frameworks and the actual field materialisation of internal control components and functions. One of the major issues relates to linking internal control with the governance and management systems of public organisations is that politically appointed personnel, public managers and staff often fail to fully grasp the true added value of internal control in improving performance and achieving institutional objectives. As discussed above, they often fail to understand that the overall role of a contemporary internal control system is much more than identifying misconduct, fraud and corruption schemes, and sanctioning individuals.

The following figure depicts four basic stages of a maturity evolution towards achieving the integration of internal control and risk management processes into the organisation’s overall governance and management systems (Figure 3.3).

Colombia has a long tradition in internal control in the public sector, but the systemic view on the different components of internal control still remains a challenge. The Colombian internal control framework (Modelo Estándar de Control Interno, MECI) was originally developed with the help of an USAID funded consultancy in 2005 and was based on Treadway’s Commission Internal Control-Integrated Framework 1992, known as COSO I. MECI has been the backbone of the internal control system and functions of Colombia until today. In 2014, DAFP undertook an in-house revision of MECI (no external consultancy involved) to address implementation challenges and better align with the updated COSO 2013 Internal Control-Integrated Framework. The new MECI is considered to be further integrated and simplified while addressing identified weaknesses of the 2005 model.

Indeed, Colombian administration seems to be quite advanced in recognising the value of streamlining internal control in public management. The Unified Management Model (Modelo Unificado de Gestión) is incorporating the internal control components and functions as integral parts of the public governance and management cycle (transversal axis). This new model is a further development of the existing Integrated Model of Planning (Modelo Integrado de Planificación, or MIP), where MECI is currently not integrated but appears as a separate pillar which aims to monitor, control and evaluate the MIP. The MIP is, together with SINERGIA developed by the Administrative Department for Planning (Departamento Administrativo de Planeación, or DNP), the most important tool to evaluate public management. However, in practice the model has been reportedly reduced to the Formulario Único de Reporte de Avances de Gestión, FURAG, a reporting tool be filled out by managers, which led to the ongoing development of the Modelo Unificado de Gestión.

Colombia should dedicate more resources in bridging the implementation gap in relation to the use of MIP and take advantage of the introduction of the new Unified Management Model, MIG, to further streamline MECI in governance and management systems. This is not an easy task and it requires first of all tailored and sustainable training modules for line management and staff as well as OCIs heads and personnel.

In Belgium, the Public Federal Service of Budget and Management Control developed an internal control framework with the goal to address the “Management gap”, that is, the difference between what was accomplished and what was planned. To this end the Management Support Unit adopted an approach that completely integrates the risk cycle and, by extension, the maintenance of the internal control system into the four phases of the management cycle (Plan – Do – Check – Act, cf. Deming), in twelve steps. Management Support created an intuitive tool, Diabolo, which serves as a process sheet and contains a complete risk module. The Belgian model has been quite successful in combining PDCA and the internal control system to reduce ‘management gaps’ and enable an organisation to improve its overall governance on the basis of feedback received on activities and results. A structured approach offers the prospect of better performance as the achievement of objectives is continually monitored, in conjunction with the use of resources.

The integration of the internal control processes to the management cycle has met quite a few challenges depending on the culture and the resources of federal institutions. It is highlighted that this is a continuous effort and each month, the Management Support Unit organises an internal control networking meeting. Through the exchange of ideas, experiences and knowledge with other institutions, the unit provides information to those who are still in the early stages of establishing an internal control system. It also offers training in internal control at the Federal Government Training Institute (Institut de Formation de l’Administration fédérale, IFA). At the request of specific services or institutions, it can also provide in-house training in the field. Furthermore, the establishment of the Federal Administration Audit Committee (Comité d’Audit de l’Administration Fédérale, CAAF) in the spring of 2010 provided a major impetus for the integration of the principles of good governance in the federal administration. Since then, the institutions within the scope of audit are required to prepare an annual report on the state of their internal control system in the previous year. The report must be submitted to the CAAF no later than February 15 of each year. These reports further constitute the basis of CAAF’s mandatory reporting to the relevant minister, as well as to the Council of Ministers. In this context, Management Support has also created a handbook addressed to institutions and, in collaboration with the secretariat of CAAF, has prepared guidelines to assist them in drawing up the report. The Belgian approach to link internal control functions with public management by using Deming’s PDCA cycle is depicted in Box 3.7.

Box 3.7. Leveraging internal control over the PDCA management cycle

A public entity’s scope and activities are determined and influenced by factors such as:

Political strategic goals
Annual policy priorities
Citizens’ expectations
Resource limitations

The Head of a public entity is accountable for managing available resources to meet stakeholder’s expectations in the most effective way. To this end he is responsible for:

Evaluating what was accomplished against what was planned
Taking action to improve the situation
Anticipating changes and possible new risks

Deming’s cycle provides for a good opportunity to illustrate the need to integrate internal control processes within the daily management operation.

The Belgian Public Federal Service for Budget and Management Control has adopted an approach that completely integrates the risk cycle and, by extension, the maintenance of the internal control system into the four phases of the management cycle (Plan – Do – Check – Act, cf. Deming), in twelve steps.

During the planning phase (Plan), the organisation defines the periodic expectations concerning the services to be provided, as well as the necessary resources. The measuring system, comprised of a set of indicators and reports, takes into account the results of the periodic monitoring.
The execution phase (Do) includes the “regular” activities of the organisation. During this phase, basic information is collected in order to be examined in the analysis phase. The management ensures the proper execution of activities and the adequate application of the measuring system.
During the analysis phase (Check), the results obtained are assessed and discussed. This is one of the most important aspects of management control; in this stage the internal control system begins to be updated based on the events that occurred during the execution phase. To this end, Management Support created an intuitive tool, Diabolo, which serves as a process sheet and contains a complete risk module. It facilitates the identification and assessment of risks. The control measures can then be evaluated, which reduces the organisation’s vulnerability to risks. Risk exposure is an indication of the possible need to deal with a priority risk.
During the reaction phase (Act), appropriate measures are developed so as to address a risk. Good support is required to ensure that the measures taken are properly implemented.

Policy-related risks have to be indicated separately because they are related to longer-term objectives in the management plan or the governmental agreement. Their monitoring requires a lower frequency than the monitoring of management risks. They can be estimated during the planning phase, by means of a SWOT analysis, with a view to possible strategic or operational rectifications. Periodic reporting from the management cycle provides a valuable contribution in this case.

Source: Public Internal Control Systems in the European Union and Practical Guide for the Development and maintenance of an Internal control System by the Belgian Public Federal Service for Budget and Management Control.

In Colombia, some concrete proposals for action could include:

Further streamlining internal control functions and activities in the governance and management systems of Colombian public administration, turning these functions in an integral component of the ongoing public administration and financial management reforms.
Take all necessary legislative and practical measures to ensure that audit recommendations are followed up and linked with the reform and the reengineering process of public management systems and administrative procedures.
Planning Units should seek the active involvement of the core operational units in the implementation of MECI and especially the risk management function.
Heads of institutions and politically appointed personnel should be going through a targeted seminar (e.g. a friendly and interactive web-based module) on the importance of internal control in improving institutional performance and achieving objectives. They have to understand that internal control is a powerful management tool that will help them succeed in their mission.
Senior and middle public managers must also learn to rely on risk management and internal audit tools for delivering their day-today operations. Internal control is neither an additional bureaucratic burden nor a “police” function. It is the means to deal with threats, scarce public resources and peoples’ mistrust to government institutions.

The criteria for the external evaluation of the internal control system could be further articulated, aligned and harmonised by strengthening the co-ordination between Comptroller General, DAFP and Contaduría

The initial analysis illustrates that the Comptroller General (CGR), DAFP and Contaduría are submitting their own reports to the legislative assembly on the effectiveness of the internal control system This activity has sometimes led to different results, creating discussions over methodological coherence instead of discussions over substance and necessary improvements. The CGR follows a methodology based on COSO II (model SICA from Chile), can make on-site visits, and focuses more on financial control; that is, the protection of public resources and fiscal related issues.

In turn, the DAFP monitors and evaluates the maturity of the internal control arrangements at the entity level by consolidating the results of the MECI self-assessment evaluation exercise which is undertaken annually by the OCIs at the entity level. The evaluation of the results of the self-assessment is raising serious challenges and there are ongoing efforts to improve the methodology, the relevant questions and indicators, as well as the documentation of the results. Indeed, self-assessment has its limitations. During the interview, it was pointed out that the self-assessment, in the past, sometimes resulted in entities being given a “green” result, but being low performing, corrupt and even facing bankruptcy. Also, for smaller public organisations, like small municipalities, the MECI evaluation can be demanding, with the result that they either contract out the evaluation, draw on already scarce resources, or respond just as a tick-the-box exercise, without providing proper documentation and evidence.

Colombia could consider formalising and standardising existing harmonisation and co-ordination initiatives by establishing a permanent working group between the external and internal control and audit institutions. There is already some progress in this field as illustrated in the 2015 evaluation exercise. The basic approach is that the evaluations by DAFP, Contaduría and CGR do not result to findings that are not meant to be compared to each other but rather, articulated and complementary. Nevertheless, there seems to be confusion between the OCIs’ personnel and the rest of the civil servants in relation to the scope, the benefit, and the validity of these different evaluation exercises.

The fundamental concepts related to internal control and risk management may be similar across the different Colombian models, but overcoming the potential learning curve to understand the differences should not be the responsibility of the audited entity. Benefits of a harmonised framework include, but are not limited to, the following:

Simplified capacity building and training of both auditors and the auditees, resulting in potential cost-savings. For instance, the cost for updating one set of standards and corresponding tools to align with evolving international updates and audit techniques would be significantly lower than doing so for different frameworks.
Easier dissemination of audit-related expertise across all three branches and levels of government, thus improving effectiveness and efficiency. The consistent application of frameworks and standards by audit entities allows for the effective and efficient application of internal control processes and risk management practices by public entities, and the bridging of policy gaps related to the design of related activities.
Streamlined self-assessment and evaluation models. Such an assessment would allow for a more reliable and effective way of monitoring and measuring the actual implementation of internal control and risk management activities. In addition, it would provide all Colombian stakeholders valuable information on addressing core issues that hinder an optimal and highly-functioning internal control system, including the risk management function.

Box 3.8 provides additional information from the European perspective for Colombian institutions to consider for improving the assessment of the internal control arrangements, including risk management and internal audit functions.

Box 3.8. Guidance for monitoring the effectiveness of internal control and risk management systems

The Guidance on the 8th European Company Law Directive on Statutory Audit offers key points for thinking about the implementation of a sound system of monitoring the effectiveness of internal control, internal audit and risk management systems. It includes the following questions:

Who monitors the adequacy of the internal control system? Are there processes to review the adequacy of financial and other key controls for all new systems, projects and activities?
- A key part of any effective internal control system is a mechanism to provide feedback on how the systems and processes are working so that shortfalls and areas for improvement can be identified and changes implemented. In the first instance if there is an internal control department, it will help managers implement sound internal controls. The operation of key controls will then be subject to review by internal and external audit along with other review agencies, both internal and external to the organisation. If no internal control department exists, guidance may be sought from risk management or internal audit.
Are arrangements in place to assess periodically the effectiveness of the organisation’s control framework?
- A key requirement of many of the internal control requirements encompassed in legislation throughout the EU and the rest of the world is an annual attestation as to the adequacy and effectiveness of the internal control system. Such an attestation should be clearly evidenced. The review of the control framework will be the responsibility of the audit committee who will receive information and assurances from internal audit, risk management and the external auditors.
Who assesses internal audit?
- The audit committee assesses the performance of the internal audit function by receiving performance information from the function itself and consulting appropriate directors and the external auditors. In addition, the function should be independently reviewed by an external agency such as the Institute of Internal Auditors (IIA), as specified in the International Professional Practices Framework, issued by the IIA.
How are the proposed audit activities prioritised? Is the determination linked to the organisations’ risk management plan and internal audit’s own risk assessment? Are the internal audit plan and budget challenged when presented?
- The work of internal audit should be set out in a risk-based plan challenged and approved annually by the audit committee. This plan should be informed by the work of other review agencies such as external audit and risk management and should contain sufficient work for the head of internal audit to be able to form an overall view as to the adequacy of the risk management process operated by the organisation. If there is no formal risk management process or if the process is flawed, then internal audit will need to rely on some other method of assessing the key activities and controls for its review. This could be based on its own risk assessment.

Source: Federation of European Risk Management Associations (FERMA) and European Confederation of Institutes of Internal Auditors (ECIIA), Guidance on the 8th EU Company Law Directive, 2011.

The DAFP could explore practical steps to improve the methodology and the implementation of the self-evaluation exercise in the framework of MECI. DAFP could work closely with other Colombian external and internal control and audit institutions to close legal and policy gaps and advance a coherent, government-wide approach to internal control and risk management by harmonising existing frameworks and improving co-ordination all competent stakeholders.

Empowering the Internal Control Offices to focus on their assurance role over the effectiveness of internal control and risk management arrangements

The benefits of the shared audit services model tailored to the needs and the capacity of the Colombian public administration could be considered and piloted in a specific policy area or at the municipality level

The real challenge for internal audit in the era of financial crisis and austerity is how to do more with less; for example, by sharing internal audit services across multiple agencies. Ideally, internal audit should place reliance upon assurance mechanisms in the first and second line of defence to target resources most efficiently on areas of highest risk or where there are gaps or weaknesses in other assurance arrangements. In many OECD member and partner countries, audit budgets are being reduced just at a time when political personnel and public senior managers need audit assurance the most.

In Colombia, there are 24 administrative sectors, 32 departments, and 1 096 municipalities. In several cases there are significant resources and capacity constraints. For example, some municipalities are just too small to have an OCI. At the same time, article 75 of law 617 from 2000 establishes that the functions of internal control and accounting may be exercised by related agencies within the respective territorial entity in relation to the municipalities of 3rd, 4th, 5th and 6th category, which practically mean that these local entities are not obliged by the law to develop their own OCI.

However, even at national level, many of the Heads of OCIs interviewed reported that they have limited personnel, that the allocated staff is not equipped with the right skills and expertise, and that they are not involved in the selection process of their personnel. Other issues relate with the remuneration regime of the OCIs Heads and staff, their position within the entities and the understanding of the importance of internal audit as an assurance function by the Heads of the institutions, the rest of the politically appointed personnel and senior public managers.

The United Kingdom, which is considered to have one of the most advanced and sophisticated internal audit functions across the world, has been working on the shared audit services approach following the publication by HM Treasury of the Financial Management review (FMR) in December 2013 (Box 3.9).

Box 3.9. The Government Internal Audit Agency development, GIAA

The GIAA is responsible for providing:

individual departmental audit and assurance services and in addition, assurance on common risks identified across Government.
internal audit and assurance policy and the development of the profession across Government (on behalf of HM Treasury).

The GIAA is currently spread over 81 locations across UK, with 67% of GIAA’s staff being outside of London, and serves 124 client public organisations. The GIAA is the home for roughly 70% of Central Government internal auditors.

GIAA’s vision is to be the primary, trusted and expert provider of consistent, high quality audit and assurance services across government that are valued by its customers and recognised as a catalyst for improvement.

Source: Presentation on “Models of Internal Audit” by United Kingdom’s Government Internal Audit Agency, Modernising Internal Audit Workshop, 05-06 December 2016.

The FMR made three recommendations concerning internal audit:

consolidate internal audit shared services, moving from the Departmental structure to a single, integrated internal audit service, which will be an independent agency of the Treasury;
strengthen the role of the head of profession for internal audit, to become “the head of government internal audit”, which will report to the director general for spending and finance in the Treasury; and
become a service to Government as a whole as well as a service to Accounting Officers.

The principle behind creating shared services is to have sufficient numbers of internal auditors grouped together for the development of capability and future leaders and the sharing of resources, particularly for specialist areas. Several heads of OCIs referred to the benefits of having the right mix of skills within their teams, and the OCI of DIAN highlighted their successful practice of putting together multidisciplinary teams of auditors. Of course the OCI of DIAN has a team of about 20–25 staff with a big audit universe of 10 000 staff and 47 sectional Directorates. They are engaging in risk based internal audit and developing assurance maps with most of the operational units. At the same time, they are also investing important efforts in showcasing the added value of internal audit and tools like ARI to senior and middle managers.

The question is whether other OCIs can also have access to sufficient resources. The shared audit services approach results in benefits deriving from the concentration of expertise, leading practices, and critical mass (e.g. concentration of fraud, forensic or cyber security experts). This model can also improve the efficiency and quality of service while reducing the financial cost, and it can adapt and evolve the audit expertise and capacity model based on the tailored expectations and needs of the beneficiaries of the services. This rationale is also driving the development of the centralised internal audit services of Belgium (Box 3.10).

Box 3.10. Belgium’s approach to centralised audit services

Key facts:

Legal framework: Royal decree (2016).
Rationale for centralisation (benchmark):
1. Efficiency & Economy (advantages of scale, specialised skills etc.).
2. Effectiveness: adequate coverage of the audit universe while taking into account Belgian federal context (small ministries).
Structure: Centralised with horizontal and vertical clusters; all federal services are covered (vertical clusters); flexible composition of audit teams;
In Progress: Developing Forensic audit methodology as a completely separated tool, only functional within IA service
Independence: Organisational independence; Reporting lines; Audit Committee; Audit Charter; Chief Audit Executive (CAE).

Methodological framework: All types of audit (operational, financial, IT, compliance, etc.); Risk-based audit approach; Audit on demand; IPPF standards of IIA

Source: Federal Public Service of Belgium, Budget and Management Control, presentation titled “Public Internal Control in the Belgian Public Administration”, 2016.

The DAFP could explore the benefits of piloting a shared audit services model in a specific policy sector, for instance in the health sector or for local governments, especially as a strategy to strengthen internal control in local areas that have been affected by the conflict, as required in the Peace Agreement.

In the health sector, there are currently eleven (11) OCIs. Health organisations sharing similar missions, tasks and processes as well as threats and challenges therefore could potentially benefit from sharing audit services. At the moment where citizens’ need to access to an effective public health care system is rising quickly, governments seek to reduce public funding while improving medical care. It is thus crucial to put in place adequate controls such as administrative, financial or broader institutional measures to mitigate not only corruption risks but also threats relating to the efficient delivery of health services. To a large extent, the healthcare sector follows regular internal control procedures, like any other sector. However, it is identified as a high risk sector, as there are many potential entry points for poor service delivery, waste and malpractice, as well as corrupt schemes and conflicts of interest (state capture, public procurement, over-billing, doctor-patient extortion to jump the treatment queue, links between medical professionals and the pharmaceuticals industry, etc.), which makes a strong case for an effective internal audit function. Public healthcare organisations are facing new and emerging risks, which require quick assessments and cost-effective mitigating measures. The complexity of the issues at stake underscore the need for a well-resourced and effective internal audit function in the health sector, with the ability to fully integrate techniques like risk-based annual audit planning, proficient use of data analytics to quickly identify patterns, trends and relationships, and IT tools to manage growing data privacy and cybersecurity concerns. In this framework, Colombia could explore how a shared audit approach could facilitate the staffing of the internal audit service in this area with sufficient technical expertise and the competency profiles to engage in this most challenging mission.

Local government is also a potential pilot area for the shared audit approach, since several local government entities do not have the size and the resources needed to develop an internal audit service. At the same time, strengthening the institutional capacity for good public management through internal control is key for the implementation of the Peace Agreement. Shared Audit Services at local level could also be interesting from the perspective of ensuring required capacities at local level for providing assurance on the implementation of the Peace Agreement. Furthermore, the fact that the heads of these entities are usually elected makes the issue of independence even more challenging than other public organisations. Municipalities and other local government entities could join forces and explore the shared audit services model to get access to the necessary internal audit expertise and assurance function that would never be able to do relying on their own resources. Article 75 of law 617 from 2000 seems to provide the legal foundation for introducing an internal audit unit delivering services to several local government institutions. The United Kingdom’s model, although not designed to cover local government, offers some valuable insights to address some of the institutional and organisational challenges of this model.

Nevertheless, in the planning phase of such an exercise, the disadvantages of this model should also be taken into consideration. One of the main issues raised is the risk of managers and staff perceiving this function as “external” control, with auditors being detached from the auditees’ daily operations and management processes. Internal audit may be perceived as an “outsider”, with limited knowledge of the operations at the single entity level.

The name of the Internal Control Offices could be changed to Internal Audit Units to reflect their core mission to provide assurance and advice and to draw a clear line from the Offices of Internal Disciplinary Control (Oficina de Control Interno Disciplinario)

One of the main challenges with having an effective institutional internal control system is to allocate roles and responsibilities across all levels of public entities in relation to internal control and risk management activities. In this framework, the assurance role of internal audit should be clear and well understood both internally and externally. Moreover, internal audit should be equipped with a modern and practical audit manual accompanied by concrete tools, like audit tailored ICT technologies, risk-based audit planning methodology as well as maturity and competency frameworks. To fulfil its mission, the internal audit has to be independent from the first and second lines of assurance, as well as the management line and associated responsibilities. Furthermore, the role of internal auditors in investigations, disciplinary procedures and complaints management should also be clarified and framed according to international leading practices and standards.

The three-lines-of-assurance model differentiates between three core functions:

Management (First Line): Functions responsible for designing, developing, implementing, and executing controls, processes, and practices to deliver services, objectives, and drive intended results (i.e., outcomes). This line may be referred to as “program management” and is responsible for the effective and efficient management of the service delivery and the daily operations of the entity. Because oversight and independent assurance cannot compensate for weak management or control, these functions generally have the greatest influence on entity-wide risk management.
Oversight (Second Line): Functions responsible for overseeing and monitoring line management and front desk activities. These groups may include (but are not limited to) functions responsible for financial control/oversight, privacy, security, risk management, quality assurance, integrity and compliance. Oversight functions also inform decision makers with objective perspectives and expertise, and provide continuous monitoring to strengthen risk management.
Internal Audit (Third Line): A professional, independent and objective appraisal function that uses a disciplined, evidence-based approach to assess and improve the effectiveness of risk management, control and governance processes. Internal Audit may provide consulting, assurance, or a combination of both to inform key decisions and support good and accountable public governance.

Figure 3.4 illustrates the basic attributes of the model adapted to the basic structure of Ministries/Departments and describes the core tasks for each of the lines.

In Colombia, there is still some confusion among public servants about the exact role and functions of the OCI. They are often perceived as a policing unit focusing on compliance. From a communication perspective and to foster the understanding of the work of the OCI among the political hierarchy, senior and line management as well as staff, OCIs could be rebranded as Internal Audit Units (Unidad de Auditoria Interna). Other options to better reflect the role of the OCIs in improving performance could include models like the one followed by the Public Company Accounting Oversight Board (PCAOB), which entrusts the internal audit responsibilities to the Office of Internal Oversight and Performance Assurance (IOPA). The IOPA is the responsible unit for providing internal examination of the programs and operations of the PCAOB and ensuring the efficiency, effectiveness, and integrity of those activities.

As previously pointed out, it is especially important to clearly define the role and the tasks of a contemporary internal audit function vis-à-vis investigations and forensic audits. These different types of oversight activities seem to create some confusion between the different Colombian stakeholders in relation to the core mission of internal audit. Figure 3.5 describes the core attributes of these oversight activities while associating theme to different levels of risk.

Concrete proposals for action to improve the capacity and raise awareness over the added value of a contemporary and robust internal audit function include the following:

Education initiatives should be undertaken for Heads of public organisations, senior and line management as well as staff on the role and importance of internal audit in promoting sound management and accountability.
Future efforts to modernise internal audit should include a communications strategy and associated tools concerning the role of internal audit and its importance in an integrated control framework for a wide audience within the Colombian public administration.
The Heads of audit units should seize the opportunity to facilitate a conversation leveraging the Three Lines of Defence Model. Internal audit can take on a consultative and educator role in helping key stakeholders understand the importance of effective three lines of defence arrangements.
Best practices of OECD member countries in terms of transitioning towards a modern integrated internal control framework demonstrate the need for a strong comptrollership function supported by a rigorous internal audit system. All efforts should be made to ensure these two functions are in place, and are supported with appropriate transition strategies.
Strengthening internal audit should be based upon a maturity model approach that involves modernising internal audit in stages, with pre-established targets for key areas including a government-wide reporting requirement to the President’s office;
The Internal Audit Manual should include concrete steps to facilitate risk-based audit planning;
The introduction of ICT tools like an IT Platform should include access to existing audits and investigations to assist in the planning and conduct of oversight activities and act as a resource tool, as well as a control to facilitate the reduction of duplicate controls and audits being conducted;
The Audit Manual should include tools to assist internal audit units to provide advisory services to management to assist them in preparing their operations to be audit ready. These tools should include Control Self Assessments, as well as strategies for undertaking audit readiness reviews.

The professional internal audit service could be improved by human resource management policies that ensure independence and by strengthening capacities of the internal audit teams through trainings in specialised topics

Civil service management practices that ensure merit, professionalism, stability and continuity in staffing are among the core prerequisites for setting up and maintaining an effective and added value internal control framework and environment.

However, in several OECD member states, budget constraints have seriously affected the ability of public sector internal audit departments to attract and retain talent, especially in technical areas such as cybersecurity and data mining. Private sector companies can obtain the most talented public sector auditors by enticing them with better pay. Given the continuing austerity measures in many government bodies globally and the culture of some public sector bodies, there is limited scope to award bonuses to those working in the public sector. Only 40% in the public sector say they have the opportunity to earn a bonus, compared to 75% for those in other sectors (Auditing the Public Sector, IIA’s Common Body of Knowledge, 2015). Colombia as well faces serious challenges in attracting, developing and retaining competent individuals with the right set of skills and ethical commitment to work in the control and audit area.

Another prerequisite for an effective internal audit function is the issue of independence and position within the organisation. With article 8 of Law 1474 from 2011, Colombia made an important step into this direction by ensuring a meritocratic selection and appointment of highly knowledgeable and skilled experts as Heads of the OCIs, involving directly the President of the Republic of Colombia.

However, in practice, it seems that in many cases the actual influence and impact of the work of the OCIs depends on the personal relationship of the Head of the OCI with the Head of the institution and his or her understanding and degree of sensitisation to the importance of internal control arrangements and especially the assurance role of internal audit. Since the Head of the OCI is formally not at the level of director, he or she does not necessarily participate in management meetings at this level, which means that there is limited information on the objectives and the challenges faced by the institutions, hampering the role of internal audit and undermining the actual impact and added value of the OCI’s work.

Therefore, Colombia could consider raising the head of the OCI to the level of a director, while at the same time giving him or her more visibility. He or she could also be given direct access to the head of the institution, and ensure that the OCI is aware of the evolution of the audit universe regarding planning, management decisions, processes, threats and opportunities and all information that are vital to its mission. Alternatively, the participation of the Heads of OCIs to the meetings at the senior management level could be formalised by regulation. Furthermore, it is key that the OCIs should be granted with enough human and budget resources (e.g. funds for training the OCIs’ staff).

Additional actions to increase the independence and showcase the importance of the OCIs role in improving governance and accountability could include:

Conducting audits and evaluations that encourage ownership, accountability and skill development among public sector managers for internal control and risk management activities.
Transferring the budget line for the remuneration of the Head of OCIs from the individual institutions to DAFP and gradually going in a single remuneration regime for all heads of OCIs. In a second phase, this could also be done for OCIs’ staff provided that this special wedge-grid would be based on concrete job and competency profiles and meritocratic appointment procedures.
Reconsider the application of the polygraph to heads of OCI, in light of the Supreme Court decision in Process 2647 of 1 August 2008, as this is contrary to the DAFP policy of building trust in the public administration and is of limited value.
Review the methodology of evaluating heads of OCI, currently based on a 360-degree evaluation, towards an evaluation based on results. This is important since the evaluation results are linked to the possibility to get a bonus payment, if more than 90 points from 100 possible points are achieved.

Concerning the capacity building and training needs, there can also be a national certification policy for internal control and audit professionals linked with training and capacity building activities. Recent reviews and relevant data from Latin America and the Middle East and North Africa (MENA) region document that there is a low percentage of practitioners who have acquired certifications like the IIA’s Certified Internal Auditor (CIA) or Certified Government Auditing Professional (CGAP). Moreover, these internationally recognised certifications have been occasionally criticised as heavily private sector-oriented, very broad and generic in relation to the specific challenges and needs of a given country, not tailored to effectively focus on core functions like public finance, public procurement and infrastructure projects, health and social welfare services.

National efforts to address the issue of weak professional expertise and capacity can include the development of customised training modules in cooperation with National Schools of Public Administration (ESAP), and/or the establishment of training centers located at the Ministry of Finance, the CGR, the PGN, Professional Chambers, Associations, or Universities. The issue of the quality of these modules and their actual impact on the skills and the performance of control and audit practitioners poses serious challenges. These efforts to develop professional “certification” limited within national context is mostly linked with hiring policies, career path, remuneration, and mobility issues in the control and audit field. Box 3.11 provides information on the Canadian example for recruiting and developing internal auditors as well as the basic attributes of a programme to train and certify public auditors applying to more than one country in south-east Europe, which worked together to address the issue of low capacity and skills.

Box 3.11. Professionalisation and capacity building of the internal audit service

A. The Canadian Internal Auditor Recruitment and Development Program (IARD Program)

I. Benefits of the Internal Audit Recruitment and Development Program.

In addition to coaching, mentoring and professional development courses, the Internal Audit Recruitment and Development (IARD) Program offers:

the experience and on-the-job training you need as you pursue a Certified Internal Auditor (CIA) designation
a development plan designed to help you succeed that includes competency-based work objectives and support from senior staff
unique on-the-job learning opportunities where you will learn the profession of internal audit in the Government of Canada
professional development sessions offered by the Institute of Internal Auditors that are related to your position and CIA certification
potential for promotion

II. Internal Audit Recruitment and Development Program work experience:

You will work under general supervision, providing support and performing assigned tasks within each of the phases of an audit engagement as a member of an audit team. Audit teams typically report to the Internal Audit Principal or the Director of Internal Audit.

Audit teams are designed to:

provide departmental senior management with opinions on the effectiveness and adequacy of risk management, control, and governance processes
report on the results of risk-based audits.

III. The Comptroller General of Canada has developed an Internal Audit Competency Profiles and Dictionary as a tool of the overarching Internal Audit (IA) Human Resources Management Framework (HRMF). The IA HRMF aims to support and enable a self-sufficient, quality IA community across the federal public sector. It provides an excellent infrastructure along with tools and support services to position the IA community as professionals who perform unique work within the Government of Canada that adds value to their organisations.

The IA competency profiles and dictionary are the main building blocks of competency-based management (CBM). They allow organisations to focus on how someone undertakes his or her job based on the skills, abilities and knowledge required to perform the work. CBM is the application of a set of competencies to the management of human resources (i.e., staffing, learning, performance management and human resources planning) to achieve excellence in performance and results that are relevant to organisations.

B. Training for Internal Auditors in the Public Sector (TIAPS)

The Training for Internal Auditors in the Public Sector (TIAPS) initiative provides an interesting example of public-sector-oriented internal audit certification that merges international best practices with localised regulatory concerns, delivered in the host country’s language.

I. Scope and key characteristics

The idea behind TIAPS started in Slovenia in 2002. The Program TIAPS was developed to strengthen qualifications in internal audit processes in the public sector, while devoting special attention to requirements introduced by the accession processes of the European Union. The mandatory and recommended guidelines issued by the IIA have long been viewed as private-sector centric and unable to comprehensively address public sector concerns.

One of the ways TIAPS addresses such gaps is to include a customisable module on legislation and taxation, written by experts from the participating country. The way in which standards and practices are taught is different from the IIA, in that it is more rules-based than principles-based. TIAPS clearly tells its students what should be done and how, as opposed to guidance issued by the IIA, which leaves generous room for interpretation.

TIAPS targets public sector employees who hold a Bachelor’s degree, and already have practical experience in areas such as accounting, financial oversight, and control. The program is composed of seven modules – divided into two levels, Certificate and diploma – of which all but the module on National Legislation and Taxation were developed by CIPFA.

II. Challenges

The biggest hurdle for implementing TIAPS is also its greatest strength; localising the curriculum. This requires involved institutions to do a lot of preparation work prior to the delivery of the program, which includes translating training material and coaching the local tutors who will deliver the content of modules in local languages.

A related issue is the need to find and hire experts to create the legislation and taxation modules. The program-implementing team engages translators with sound knowledge of material substance, and the initial translation is checked by an editor/proof-reader, to make any necessary language revisions, in line with standard terminology in each respective country.

Despite being a relatively young program, TIAPS provides specialisations. These, however, have yet to achieve the total level of equivalence to directly replace specialised certifications – such the Certified Information Systems Auditor (CISA), provided by the Information Systems Audit and Control Association (ISACA) – though there are plans of doing so in the medium term.

The program also does not have a way to monitor and ensure that its certified practitioners keep up-to-date with evolving audit trends, which both IIA and ISACA do, through their continuing professional education requirements.

Source: Office of the Comptroller General of Canada, IARD Post-Secondary Recruitment; https://emploisfp-psjobs.cfp-psc.gc.ca/psrs-srfp/applicant/page1800?poster=941922&toggleLanguage=en; IARD Program; http://www.tbs-sct.gc.ca/ip-pi/job-emploi/ford-rpaf/benefitsiard-avantagesrpai-eng.asp; Training for Internal Auditors in the Public Sector-An Alternative Approach for State Internal Auditors, Knowledge Showcases, Asian Development Bank, 2016.

In order to strengthen the professional audit function and ensure the delivery of robust assurance services, Colombia could consider implementing the following proposals for actions:

A mandatory training program for all current and future internal auditors (and investigators) should be introduced in partnership with University institutions, the National School of Public Administration and professional organisations like the Colombian Institute of Internal Auditors.
Future hiring of internal audit positions should place a premium on individuals holding a Certified Internal Auditor (CIA) or Certified Government Auditing Professional (CGAP) designation or a similar relevant designation such as accounting (e.g. CPA, CA, CGA, CMA).
Consideration should be given to facilitating IIA membership for all internal auditors to promote self-study and professional development as well as facilitate the obtainment of the Certified Internal Audit designation
In order to enhance the independence of audit professionals, as well as ensure their level of qualifications, alternative staffing measures should be assessed and implemented. Among the options that should be considered are fixed-term appointments and/or some form of registry of control, risk and audit professionals whose entry would be based upon the necessary designations and skills.
The Internal Audit Manual should include a proposed competency model, along with sample job descriptions.
Ensure that induction trainings are covering knowledge on the institution and the sector, and that additional training opportunities to achieve certification are available.
Ensure continuous training opportunities for the head and the staff (Colombia had an absolute increase of 58%, in just 6 months, of people acquiring the CIA designation).
ESAP with national IA and ACFE chapters could develop tailored modules to prepare auditors for the Certified Internal Auditor (CIA) and the Certified Fraud Examiner (CFE) exam.
Ensure a clear staff profile, in relation with entity size, and expert staff in the internal audit units (OCI):
1. At national level, heads of OCI (126 in total) are selected based on merit and are directly appointed by the President, however their staff is not subject to clear job and competency profiles, and meritocratic selection.
2. Therefore, whether the staff of the audit unit is adequate strongly depends upon the willingness of the head of the institution. The work of the head of the OCI can therefore be easily either supported or undermined.
3. The OCIs should be involved in the selection process of their personnel together with the HR Units.

Colombia could explore the benefits from the introduction of independent audit and risk boards or committees in relation to the effectiveness and the efficiency of internal control and risk management functions

In Colombia, there are currently no Audit and Risk Committees or equivalent bodies established in public organisations. According to relevant data, Audit and Risk Committees are established in the vast majority of private sector organisations but they are much less common in public sector organisations, such as line ministries and local government authorities. State Owned Enterprises (SOEs) usually have to introduce an Audit Committee or equivalent body complying with the needs of the market or corporate governance regulations. The G20/OECD Principles for Corporate Governance (July 2015) clearly state that the Board is responsible for “Ensuring the integrity of the corporation’s accounting and financial reporting systems, including the independent audit, and that appropriate systems of control are in place, in particular, systems for risk management, financial and operational control, and compliance with the law and relevant standards.”

It is considered good practice for the internal auditors to report to an independent audit committee of the board or an equivalent body which is also responsible for managing the relationship with the external auditor, thereby allowing a co-ordinated response by the board. In certain OECD member countries, for example, the USA and New Zealand, the existence of audit committees has caused top management to focus on internal control and risk management, and has attracted the attention of the senior management to the role of the internal auditor. Furthermore, in the private sector, an audit committee would be typically charged with overseeing the internal and external financial reporting processes, risk management, internal control, compliance, ethics, the external audit arrangements and ensuring the independence of internal audit function. (See relevant guidelines in IIA, INTOSAI, COSO-Treadway Committee and the International Federation of Accountants).

Moreover, a truly independent audit/risk committee with high expertise can harness the political influence on control and audit activities and mitigate the potential bias of auditors assessing the quality of the internal control and risk management arrangements. It can also strengthen the impact of these processes inside the organisation, linking them to the achievement of the entity’s objectives, thus facilitating the involvement of the middle and line managers and the rest of the personnel.

The following box demonstrates some leading examples across OECD member states in establishing Audit and Risk Committees/Boards in public organisations (Box 3.12).

Box 3.12. Leading examples of Public Audit and Risk Committees/Boards

The Canadian Departmental Audit Committee

The Departmental Audit Committee (DAC) is a strategic resource for the deputy head. It provides objective advice and recommendations to the deputy head regarding the sufficiency, quality and results of assurance on the adequacy and functioning of the department’s risk management, control and governance frameworks and processes (including accountability and auditing systems). Deputy heads can use this information to enhance accountability, transparency and the overall performance of their departments. Within the Canadian federal public administration, the independent Departmental Audit Committees with external members have become essential advisors on Risk and Internal Controls design and assessment.

Audit committees must include a majority of external members recruited from outside the federal public administration, i.e. non-government employees, contractors, ministerial appointees
Appointed for a term not exceeding four years, which may be renewed for a second term
Have, at a minimum, three members with a quorum of a simple majority
Since 2007, over 350 members appointed to 42 Departmental Audit Committees (DACs)

The United Kingdom’s Audit and Risk Assurance Committee:

Following the 2011–2013 public internal control reform, Audit Committees are now usually termed Audit and Risk Assurance Committees. The Audit and Risk Assurance Committee, made up of independent non-executive directors, supports the Accounting Officer with the primary responsibility of reviewing the comprehensiveness and reliability of assurances throughout the year.

The Audit and Risk Assurance Committee plays a key role in ensuring that management’s response and resolution of audit recommendations and identified risks is satisfactory. Audit and Risk Assurance Committee responsibilities are set out in the Audit and Risk Assurance Committee Handbook published by the Treasury. The Committee has particular responsibilities relating to the work of internal and external audit and to assurance and financial reporting issues.

There is consequently a major synergy between the purpose of the Head of Internal Audit and the role of the Audit and Risk Assurance Committee. The committee will typically be interested in internal audit’s charter/terms of reference to ensure that it has sufficient status and independence to operate freely and effectively in its work. It will also take a close interest in the adequacy of audit resources. The committee will advise the board and Accounting Officer on internal audit strategy and plans, forming a view on how well they support the Head of Internal Audit’s responsibility to provide an annual opinion on the overall adequacy and effectiveness of the organisation’s governance, risk management and control processes.

Italy’s approach to Audit and Risk Committees:

In Italy, the legislative decree No 150/2009, which implemented Law No 15 of 4 March 2009 on improving the productivity of the public sector and the efficiency and transparency of public administrations, set up two bodies to measure and appraise the organisational and individual performances of public administrations:

A central body known as CIVIT (Independent Commission for the Appraisal, Integrity and Transparency of Public Administrations) and,
For each individual administration, the OIVs (Independent Performance Evaluation Bodies).

The law tasks CIVIT, which is called upon to show independence of judgement and evaluation and work in complete autonomy, with the task of directing, co-ordinating and supervising the appraisal functions to ensure the transparency of the systems adopted and the visibility of the indicators of public administrations’ management performance. This function is also particularly relevant, because the law sees data transparency as a tool for ensuring the integrity of public administrations and thus preventing the serious problem of corruption. The cabinet appoints the members of CIVIT.

Each administration also has an Independent Performance Evaluation Body (OIV) that performs a multitude of tasks such as:

monitoring the overall operation of the system of evaluation, transparency and integrity of the internal controls and drawing up an annual report on its working;
promptly reporting any problems to the relevant internal government and administration organs;
ensuring that the measuring and evaluation processes are correct in order to uphold the principle of rewarding merit and professionalism;
applying correctly the guidelines, the methods and the instruments provided by CIVIT;
promoting and certifying transparency and integrity; and
checking the results and good practices arising from the promotion of equal opportunities.

Source: Office of the Comptroller General of Canada, Departmental Audit Committees, http://www.tbs-sct.gc.ca/hgw-cgf/oversight-surveillance/ae-ve/dac-cmv/index-eng.asp and Compendium of the Public Internal Control Systems in the EU Member States, 2014, http://ec.europa.eu/budget/pic/lib/book/compendium/HTML/index.html

The introduction of Audit and Risk Committees however poses several challenges including selection, appointment and remuneration issues, however. Furthermore, the exact scope and institutional relations with existing control and audit stakeholders have to be carefully examined.

Some concrete examples and steps forward could include:

DAFP could explore the opportunity of introducing management-led Audit Committees, including draft committee terms of reference as a tool to assist in promoting audit independence.
The OECD member states experiences – and especially the Canadian example of dedicated Departmental Audit Committees for large entities and shared AC for smaller entities within the same policy field – could be a good starting point for this discussion.
DAFP could consider piloting an Audit and Risk Committee to assess the impact and the added value in areas like enhancing the independence of internal audit, raising awareness over the importance of the assurance function, and securing adequate resources. Ann independent assessment could also be provided, as well as making recommendations as needed on the capacity, independence and performance of the internal audit function

Proposals for Action

Therefore, the following actions could be taken by Colombia to strengthen integrity by mainstreaming internal control and risk management into the public governance systems.

Embedding fraud and corruption risk management in Colombian public organisations

The DAFP should further engage in a risk-based approach as the bedrock to consolidate a control environment that is non-conducive to fraud and corruption.
The integrity attributes of the internal control environment need to be strengthened and the right tone at the top demonstrated to create the necessary preconditions for effectively managing fraud and corruption risks.
The concrete role and responsibilities of Internal Control Offices in preventing, detecting and responding to fraud and corruption schemes need to be better defined.
The use of data analytics and big data could be further explored and leveraged to strengthen transparency and support a pre-emptive risk-based approach to tackle fraud and corruption.

Integrating internal control for good and accountable public governance

DAFP should further work on bridging the implementation gap and mainstreaming internal control functions in the management systems of public organisations.
The criteria for the external evaluation of the internal control system could be further articulated, aligned and harmonised by strengthening the co-ordination between Comptroller General, DAFP and Contaduría.

Empowering the Internal Control Offices to focus on their assurance role over the effectiveness of internal control and risk management arrangements

The benefits of the shared audit services model tailored to the needs and the capacity of the Colombian public administration could be considered and piloted in a specific policy area or at the municipality level.
The name of the Internal Control Offices could be changed to Internal Audit Units to reflect their core mission to provide assurance and advice and to draw a clear line from the Offices of Internal Disciplinary Control (Oficina de Control Interno Disciplinario).
The professional internal audit service could be improved by human resource management policies that ensure independence and by strengthening capacities of the internal audit teams through trainings in specialised topics.
Colombia could explore the benefits from the introduction of independent audit and risk boards or committees in relation to the effectiveness and the efficiency of internal control and risk management functions.

References

AICPA (2003), “Consideration of fraud in a financial statement audit (supersedes SAS No.82)”, Statement on Auditing Standard (SAS), No. 99, American Institution of Certified Public Accountants, New York, NY.

ACFE (2014), Report to the Nations on Occupational Fraud and Abuse, Association of Certified Fraud Examiners, Austin, TX.

ACFE (2016), Report to the Nations on Occupational Fraud and Abuse, Association of Certified Fraud Examiners, Austin, TX.

ANAO (2015), Public Sector Audit Committees: Independent assurance and advice for Accountable Authorities, Australian National Audit Office, Canberra, www.anao.gov.au/work/better-practice-guide/public-sector-audit-committees-independent-assuranceand-advice.

Australian Standard (2008), AS 8001–2208, Fraud and Corruption Control, Sydney, Australia.

Belgian Public Federal Service of Budget and Management Control (2012), Practical Guide for the Development and Maintenance of an Internal Control System, Management Support Unit, Brussels, Belgium.

Committee of Sponsoring Organizations of the Treadway Commission (COSO) (2013), Internal Control-Integrated Framework.

Committee of Sponsoring Organizations of the Treadway Commission (COSO) (2004), Enterprise Risk Management.

Committee of Sponsoring Organizations of the Treadway Commission (COSO) and Institute of Internal Auditors (IIA)), Leveraging COSO Across the Three Lines of Defence.

Crime and Misconduct Commission of Queensland (2005), Fraud and Corruption Control – Guidelines for Best Practice, Brisbane, Australia.

European Commission (2014), Fraud Risk Assessment and Effective and Proportionate Anti-Fraud Measures, http://ec.europa.eu/regional_policy/en/information/publications/guidelines/2014/fraud-risk-assessment-and-effective-and-proportionate-anti-fraud-measures

European Commission, (2014), Compendium of the Public Internal Control Systems in the EU Member States, Second Edition, http://ec.europa.eu/budget/pic/lib/book/compendium/HTML/index.html

Institute of Internal Auditors (2016), International Professional Practices Framework, Practice Guide: Internal Audit and the Second Line of Defense, Altamonte Springs, Fla., USA.

Institute of Internal Audit Research Foundation (2015), The Global Internal Audit Common Body of Knowledge, Responding to Fraud Risk – – Exploring Where Internal Auditing Stands, Altamonte Springs, Fla., USA.

Institute of Internal Audit Research Foundation (2015), The Global Internal Audit Common Body of Knowledge Auditing the Public Sector – Managing Expectations, Delivering Results, Altamonte Springs, Fla., USA.

Institute of Internal Auditors (2014), Global Public Sector Insight: Independent Audit Committees in Public Sector Organizations, Altamonte Springs, Fla., USA.

Institute of Internal Auditors (2013), The Three lines of Defence in Effective Risk Management and Control, IIA’s Position Paper, Altamonte Springs, Fla., USA.

Institute of Internal Auditors Audit Executive Center (2012) Assurance Mapping – Charting the Course for Effective Risk Oversight.

Institute of Internal Auditors Research Foundation (2011), Internal Auditing’s Role in Risk Management, The IIARF White Paper, Altamonte Springs, Fla., USA. https://na.theiia.org/iiarf/Public%20Documents/Internal%20Auditings%20Role%20in%20Risk%20Management.pdf.

Institute of Internal Auditors (2011), Global Technology Audit Guide (GTAG), IPPF – Practical Guidance, Data Analysis Technologies, Altamonte Springs, Fla., USA.

Institute of Internal Auditors Research Foundation (2009), Global Technology Audit Guide, Fraud Prevention and Detection in an Automated World, Altamonte Springs, Fla., USA.

International Organisation of Supreme Audit Institutions (2010), “Guidelines for Internal Control Standards for the Public Sector”, INTOSAI Guidance for Good Governance, Vienna, Austria. http://psc-intosai.org/data/files/9A/87/E1/E2/1E927510C0EA0E65CA5818A8/INTOSAI-GOV-9100_e.pdf.

Noel Hepworth and Robert de Koning (2012), “Audit Committees in the Public Sector. A Discussion Paper”, London-Brussels. https://www.pempal.org/sites/pempal/files/attachments/Audit%20committees%20in%20the%20public%20sector%20May%202012.pdf.

OECD (2015), G20/OECD Principles of Corporate Governance, OECD Publishing, Paris https://doi.org/10.1787/9789264236882-en

OECD (2015), “Budget reform before and after the global financial crisis”, Working Paper, OECD Publishing, Paris, GOV/PGC/SBO(2015)7.

OECD (2015), Recommendation of the Council on Budgetary Governance, OECD Publishing, Paris.

OECD/IDB (2014), Government at a Glance: Latin America and the Caribbean 2014: Towards Innovative Public Financial Management, OECD Publishing, Paris. https://doi.org/10.1787/9789264209480-en

OECD (2013), OECD Integrity Review of Italy: Reinforcing Public Sector Integrity, Restoring Trust for Sustainable Growth, OECD Public Governance Reviews, OECD Publishing, Paris. https://doi.org/10.1787/9789264193819-en

OECD (2012), Integrity Review of Brazil: Managing Risks for a Cleaner Public Service, OECD Publishing, Paris. https://doi.org/10.1787/9789264119321-en

OECD (2009), Corporate Governance and the Financial Crisis: Key Findings and Main Messages, OECD Publishing, Paris, www.oecd.org/corporate/ca/corporategovernanceprinciples/43056196.pdf

Office of the Comptroller General of Canada (2011), Internal Audit Sector, Internal Audit Talent Management – Competency Profiles and Dictionary, Treasury Board of Canada Secretariat, Ottawa, Canada.

Public Internal Control in the European Union, PIC EU–28 Conference (2015), Assurance Maps, Paris.

United Kingdom’s HM Treasury (2013), Review of Financial Management in Government, London, UK.

United Kingdom’s HM Treasury (2012), Assurance Frameworks, London, UK.