2. Evidence-based enforcement

This chapter explains the rationality and purpose of the evidence-based enforcement principle. It aims at identifying evidence of the principle within OEFA’s daily operation and work. This includes an assessment of OEFAs’ implementation of inspection duties, the use of sources of information and indicators, etc. as supporting promoters of regulatory decisions.

    

Regulatory enforcement and inspections should be evidence-based and measurement-based: deciding what to inspect and how should be grounded in data and evidence, and results should be evaluated regularly (OECD, 2014[1]), (OECD, 2018[2]).

Enforcement and inspections need to be evidence-based; thus, actions of regulatory agencies and their performance should be evaluated according to a set of well-defined indicators and reliable data. Collecting data on activities and outputs (e.g. how frequently an agency conducts inspections, how many entities are subject to inspections, how much time, private or public is taken up with inspections – and what are the administrative sanctions or criminal prosecutions that may follow) is necessary to assess the use of resources and burden creation on businesses. However, these indicators should not be taken as a reflection of the effectiveness (or lack thereof) of an agency.

In new regulations, enforcement planning (including inspections) should be covered during the impact assessment process. Likewise, evidence-based enforcement should be “anchored” during both design and ex post review of regulation. Moreover, it is necessary to analyse whether inspections and enforcement are needed for a proposed regulation, as well as the organisation scheme, the resources and methods to follow – these issues should also be considered when reviewing regulations ex post. In this process, it is of particular relevance, inputs from inspectors and stakeholders, which can offer valuable information.

The mandates of institutions in charge of regulatory enforcement and inspections should reflect goals in terms of risk reduction and pursuing public interest. Institutions empowered to inspect and enforce should have a purposes in terms of the harm they will mitigate and the positive outcomes they will contribute to achieve.

The reliability of the data used is of particular importance. As a rule, data collected directly from an agency’s processes should not be used to assess compliance levels, as it is by no means “independent” and distorts incentives (e.g. number of prosecutions or sanctions). For instance, the agency can seek to increase or decrease these indicators to “perform” better, without reflecting an actual improvement in compliance levels.

For each regulatory enforcement agency or structure mandated with the authority to conduct enforcement and compliance, governments should ensure that this mandate is clearly defined with reference to the outcome indicators that the agency aims to influence (e.g. number of preventable deaths and injuries due to specific hazards, amongst others), and that the agency is required to track and report on these regularly.

Agencies should collect the data according to strictly defined protocols. In cases when data is produced or collected by the agency itself, it should be regularly crosschecked by independently conducted representative surveys. In all cases, governments should publish all the relevant information pertaining to indicators, how they were defined (and based on which assumptions and logic) and how they are measured.

Evidence-based enforcement also means that available evidence should be used for operational purposes (in particular, inspections’ planning, and selection of the inspection-related activity). For new regulatory agencies that are still in the process of gathering empirical data, it is crucial that processes and methods are designed to make the best possible use of evidence, as soon as it is available, and that there is a real understanding of what ‘evidence’ means.

OEFA’s use of evidence-based regulation and strategic principles

The importance of evidence-based inspections and enforcement is stated in OEFA’s regulations and strategic principles. In recent years, there has been a real effort to embed this principle in policy or strategic documents, and that this then cascades through regulations to ensure its implementation.

The most representative examples are the current Institutional Strategic Plan and the Regulation on Inspections of OEFA, which underline the link between risk management and the need to ground OEFA’s activities on evidence. These documents also indicate the need to regularly measure its performance and assess whether activities need to be adjusted to improve results.

The Institutional Strategic Plan establishes as the first direction for the institutional policy ‘to ensure the prioritisation of actions based on evidence’. The declaration of institutional policy and the presentation in the PEI also emphasise the need for OEFA to perform activities “in line with a risk-based approach and based on evidence”. OEFA’s Regulation on Inspections’ Principles include ‘evidence-based inspection’, defined as the need for inspection actions to be planned, performed and concluded taking into account objective information gathered by the inspection authority.

These concepts are broadly taken up and communicated by OEFA’s management and generally reflected in the tools developed by OEFA. However, the review suggests that further efforts are needed to ensure that the concept of “evidence-based” is fully understood and implemented throughout the enforcement and inspection system of environmental regulation in Peru. “Evidence-based” should mean that regulatory inspection and enforcement actions are decided and evaluated “against a set of well-defined indicators, and based on reliable and trusted data”, as explained in Principle 1 of OECD’s Regulatory Enforcement and Inspections.

Implementation of the principle in planning of inspection-related activities

One of the main tools to govern the operations of OEFA and EFAs (Entidades de Fiscalización Ambiental)1 is the PLANEFA – an annual plan of inspections to be carried out by each relevant authority. The PLANEFA should include all inspections planned according with the competences of the relevant EFA. It is to be noted that not only “regular” (proactive) inspections are included in the PLANEFA, but it also comprises a share of inspections to respond to emergencies, complaints or requests from other authorities. In addition, other supervision-related activities are also contained in the PLANEFA, such as ‘environmental evaluations’, which are defined as “actions of surveillance, control and monitoring to determine the quality of the environmental components, that an EFA carries out within its area of competence” (according to article 4 of PLANEFA Guidelines) (OEFA, n.d.[3]). Other document that underpins the inspection process by OEFA’s regulation on inspections is the Bylaw on Supervision, where inspections should be evidence-based. Furthermore, the piece of legislation covers the responsibilities of inspectors and supervisor, defines the types of supervision and the results of the latter. Additionally, it describes the administrative measures that the supervision authority can use in case of non-compliance and the resources that regulated entities have to challenge the decision.

As per the relevant “Orientation Guide” (or Handbook, in English) elaborated by OEFA, the inspections’ planning shall be determined by identifying the environmental ‘issue’ considered as significant (step 1 out of 8 of the proposed logical framework methodology), and by highlighting the ‘problem or risk’ (step 2 of 8) (OEFA, 2019[4]). The PLANEFA Handbook is a useful tool prepared by OEFA to ensure that all EFAs develop their PLANEFA following the same method. It explains systematically (including figures and graphs), which elements should be taken into account, in which order, how it must be used, what are the deadlines are, amongst others. Based on the specific competences of each EFA, the latter shall identify the environmental “issue” (problemática) considered as significant. Then, based on the identified “issue”, the EFA must ascertain that the “problem” actually exists, and whenever possible, measure its magnitude or existing environmental risk. This involves reviewing the sources that demonstrate that the “problems or risks” exist.

This methodology would benefit from some improvements. This would include further clarifying the concepts included – e.g. using internationally accepted notions in concepts as “hazard” instead of “issue”, and “risk” instead of “problem or risk”. Including in-depth explanations and training on this would be advisable. As of today, there seems to be need for deeper clarification about the meaning and content of these, and related terms. In many cases observed during the review, especially in the case of local EFAs, the ‘issue’ to be first identified corresponds to the field of competence of an EFA that requires the most attention. Following this, the EFA searches empirical evidence to prove that such topic actually must be seen as a priority. From a good international practice perspective, the methodology to define the planning of inspection-related activities would need to be inverted. Evidence should be the initial empirical element on which a topic can be defined as relevant or risky, and not the other way round, i.e. justifying that a topic creates significant risk, once it has already been chosen (based on existing structures).

Main evidence sources for inspections’ planning

The PLANEFA Handbook includes four different evidence sources to determine the relevant environmental “issue”: citizens' perceptions, results of previous environmental enforcement, environmental management tools (determined during the licensing process of an establishment), and other sources (such as environmental liabilities, economic activities performed without environmental license, studies revealing existing environmental problems).

In practice, a heavy share of inspections are performed by OEFA based on reactions to complaints and socio-environmental conflicts, rather than on a prior assessment of risk. Observations made during the fact-finding mission in regions show that this reliance on complaints and conflicts is also very clear in local EFAs visited.

OEFA itself plans its activities and measures based on the risk level assessed through four main sources of evidence: a) dialogue with stakeholders in areas of socio-environmental conflicts, b) results of previous actions by OEFA, c) citizen’s perceptions of environmental quality and d) prioritised information from other public agencies – (OEFA, 2018, p. 8[5]). While inspections’ record and history are a recognised criteria widely used in good international practice and to be taken into account for planning of supervision-related activities, the use of the other sources of evidence mentioned above as main criteria raises questions.

While socio-environmental conflicts and complaints are factual in a sense (they are reported and recorded), it should be defined with certainty whether they represent real sources of harm or risks to the environment, as they usually reflect a mix of realities and perceptions, which can only be determined through the collection of hard evidence. Also, it is to be noted that the criterion “prioritised information from other public agencies” might be another form of complaint or channelling of conflicts.

Socio-environmental conflicts as source of evidence

The office of the Ombudsperson reports on a monthly basis, the existing social conflicts and their status. Over 79% of these conflicts in Peru are reported as being of socio-environmental nature (Defensoría del Pueblo, n.d.[6]). The relevant report issued in June 2019 lists 117 socio-environmental conflicts, out of which 87 are “active” (not yet solved) (Defensoría del Pueblo, n.d.[6]). In general, socio-environmental conflicts are generated by the grievances of communities over economic activities, as mining or hydrocarbons production and extraction, which cause pollution and affect negatively their own activities (e.g. agriculture, cattle breeding) and the health of their inhabitants. Politically speaking, these conflicts are extremely sensitive and create the need for various authorities, including OEFA, to intervene determining if the perception of harm or likely harm is correct.

OEFA chose 33 of the existing socio-environmental conflicts reported by the Ombudsperson’s office to give attention during the calendar year 2019. In order to support this task, OEFA created an open, online tool designed to provide an overview of the main characteristics of the existent socio-environmental conflicts and their status with regard to OEFA‘s intervention. The OEFA’s tool operates with information for the whole country at department and region’s level publishing the existence of protected natural areas, the number of population centres, degraded areas, complaints, regulated entities and OEFA’s actions by sector (infrastructure, industry, mining, hydrocarbons, agriculture, amongst others) (OEFA, n.d.[7]).

The reason why socio-environmental conflicts are considered by OEFA as a prioritisation criterion is that these are one of the main barriers to the development of extractive activities, which are of crucial importance for the Peruvian economy. The results of inspection-related activities carried so far in these areas show that the perception of pollution has often been confirmed–i.e. the quality of water, air, soil is far from ideal.

The activities performed by OEFA have allowed shedding light on the causes for existing contamination, which were unknown to local communities, and the Peruvian State. Investigations carried out show that contamination has been caused by both natural and/or anthropogenic causes. In the latter, immediate measures to manage or mitigate risks have been enforced, accompanied by corresponding sanctions. As of today, over 200 administrative measures have been imposed (OEFA, n.d.[7]). Based on the results following inspection-related activities performed in socio-environmental conflict areas, in most cases OEFA found that in fact, there was a high-impact on water, air, soil, etc. Based on this trend, OEFA concluded that the probability of a harm occurring is very high.

While socio-environmental complaints are evidently a relevant source in terms of assessing the likelihood of a problem, they give only limited information (or information of limited reliability) about the potential magnitude or severity of the problem. For OEFA’s supervision activities to be properly ‘evidence-based’, complaints information should be systematically combined with other sources. For example, an analysis of the actual impact of past violations, the potential effects of current economic activities, and the reliability of past complaints in terms of predicting the severity and magnitude of damage. OEFA thus, would need to further develop its assessment criteria to better differentiate between different conflict situations based on the quantifiable environmental impact.

It should also be pointed out that socio-environmental conflicts feature strongly among the criteria used by OEFA to define the level of risk of regulated establishments in a certain sectors – and subsequent planning of activities. Just to name an example, risk criteria used to plan inspection-related activities in the mining area are the following: a) mining exploration at risk of socio-environmental conflicts; b) high-risk cases for effluent discharge points into water bodies; c) presence of socio-environmental conflict; d) files with alleged infractions (subject to potential sanctions) entailing moderate and significant risk (OEFA, 2018[5]). In this case, socio-environmental conflicts are present in two out of four criteria used to plan inspection and enforcement activities. Complaints are also among the risk criteria to take into account for inspection planning for year 2019 in the solid waste area (OEFA, 2018[5]).

Complaints as source of evidence

Both the General Law on Environment (Art. 43) and the Single Consolidated Text (TUO) of the Law on General Administrative Procedure2 (Art. 116) require OEFA to address any complaints or grievances it receives. In particular, Art. 43.1 of the General Law on Environment foresees that “Public entities must establish in their Regulations on Organization and Functions, or other documents of management, procedures for the care of those complaints and ways of communication to the public, according to the parameters and criteria set by the Ministry of Environment and under the responsibility of its highest representative. The entities should send annually a list of complaints received and the solutions found […]”. Art. 116.3 of TUE foresees that “Its filing [of a complaint] requires carrying out the necessary preliminary proceedings and, once its credibility has been proven, to initiate the respective inspection ex officio. The rejection of a complaint must be motivated and communicated to the complainant […]”.

Based on this legal duty, OEFA has been undertaking efforts to develop a system to classify complaints received based on their characteristics, evidence submitted by the complainant, amongst others, in order to define the resources that need to be allocated to deal with the specific complaint. This classification allows classifying the complaint as “high risk”, “medium risk”, “low risk”, “or no existing issue”. According to the result of the assessment, immediate action will be taken by OEFA (high risk), the information will be used to plan subsequent inspection-related activities (e.g. during next calendar year – medium risk), or no action will be undertaken. Figure 2.1 below shows the results of this assessment. For the time being 44% of complains analysed by OEFA in 2019 resulted – or will result – in an immediate inspection.

Sinada, the online tool that allows filing an environmental complaint is under improvement. One of the new functionalities is the interconnection with other existing registries (OEFA, n.d.[8]). OEFA’s management acknowledges in any case the need to perform an ex post evaluation of the complaints management system to improve and refine the assessment criteria currently used. A first ex post assessment was performed on the implementation of Sinada rules over the period 2009-2017 (OEFA, 2019[9]).

Figure 2.1. Management of complaints received by OEFA through Sinada

Source: OEFA (n.d.[10]), Organismo de Evaluación y Fiscalización Ambiental, https://www.gob.pe/minam/oefa (accessed 14 October 2019).

Measurements, empirical data and inherent risks as evidence to support decisions

A wide range of useful and meaningful data and measurements on operators, pollution, etc. is now being collected through “preventive environmental assessments”, technical studies, automatic monitoring networks and information gathered during inspections.

The “preventive environmental assessment” is a meaningful inspection-related activity in terms of data collection and risk prevention. The procedure associated with such environmental assessment has seven steps; it includes the co-ordination, invitation and inclusion of interested stakeholders, as well as the organisation of events a) to propose the plan for the monitoring activity and b) to deliver of the findings and results of the activity. OEFA organises these “participatory environmental monitoring activities” whenever there is “environmental sensitivity”, a socio-environmental conflict, or other criteria considered within the relevant PLANEFA.

Data collected since 2017 through preventive environmental assessments, technical studies, automatic monitoring networks and information gathered during inspections are fully incorporated into the INAPS (Direct Inspection system). The INAPS is an information tool aimed at allowing to manage information on inspections performed on regulated subjects from the different economic sectors. The INAPS also contains various documents and material produced as a result of inspection activities. However, information on supervision-related activities performed under previous management of OEFA still needs to be reconstructed (for a more detailed description of IT tools developed by OEFA and use of data, see Chapter 9 on Information integration.

Additional consolidated data and information management systems would still be useful to make more evidence-based strategic and operational choices. However, the review shows that there are pending efforts at using the existing and collected data to full potential. For example, analyse past inspection results, monitoring data, assessments, measured impacts, amongst others, to plan activities and thus gradually relying less on reports of conflict and complaints. These are necessary steps to ensure that the inspection and enforcement framework and practices in the environmental sector of Peru evolves into a system fully based on evidence, leaving perception elements behind.

Use of indicators focusing on outcomes to inform choices

OEFA has developed a large number of indicators to assess the performance of regulatory inspections and enforcement (included in the PLANEFA and the current Strategic Plan). However, the profusion of indicators can lead to certain confusion and further explanations to ascertain which indicators are actually being used are required. In particular, whether those actually measuring the effectiveness of the authority in terms of outcomes are effectively used, as most of the numbers shown in OEFA’s annual report are output results (activities performed), instead of outcomes in terms of risk reduction or environmental harm (OEFA, n.d.[11]). In addition, the performance of EFAs is evaluated based on the completion of the PLANEFA, which includes whether the inspections planned for the following year were actually performed, as well as indicators aimed at measuring institutional strengthening of the EFAs.

OEFA has reported that most of outcome indicators as those set in OEFA’s PLANEFA for 2019 have not yet been measured – and will be measured starting from the end of this calendar year.

OEFA’s PLANEFA for 2019 presents a series of indicators developed for each function of the organisation, as shown in Table 2.1.

Table 2.1. List of indicators in OEFA's PLANEFA 2019
Function	Activities	Products	Results	Impacts
Evaluator	No. Population surveys. No. Participatory workshops	Early environmental assessments	% improvement in citizen perception regarding the activities that are carried out	USD investment is made in a healthy environment and in harmony with the surrounding populations
	No. Specialised technical studies No. Diagnostics of environmental components	Environmental assessments that determine causality	% sources of affectation identified	% evaluation actions serve as input for the control of critical components or control stage
	No. Monitoring	Environmental monitoring reports	Number of days of alerts reported to deviations found in environmental monitoring
Supervisor	No. Special supervision actions by sector/topic	No. Supervisory records concluded	% compliance with environmental obligations	% of decrease in concentration of contaminants in receiving bodies
	No. Regular supervision actions by sector/topic	% compliance with scheduled supervision	% control of highly identified components	Number of people directly benefited by the control of environmental risk
		% compliance with scheduled supervision		Improvement of perception of the population in the areas of influence of the activities carried out
Supervision and sanctioning	No. Resolution on initiation of sanctioning procedure	No. Cases sanctioned concluded	% compliance with Environmental obligations (by correction or cessation during the sanctioning procedure)	No. of hectares with achievement of compliance and/or cessation of conduct
	No. Final reports of instruction	% compliance with programmed sanctioning procedure		Millions of USD in economic value recovered on ecosystem services
	No. Director resolutions	-		Number of people directly benefited by the correction or cessation of behaviour
Function	No. of regulatory problems identified	No. Regulatory proposals made	% regulatory proposals approved	Regulatory problems identified solved by the legal instrument
(OEFA, 2018[5]), Plan Anual de Evaluación y Fiscalización Ambiental del OEFA – PLANEFA 2019, https://www.oefa.gob.pe/planefa/planefa-2019 (accessed 14 October 2019).

Such indicators have been developed reflecting activities, products, results and impacts of OEFA’s activities. An in-depth description of indicators developed and used by OEFA is provided in Chapter 13.

Assessment

Both OEFA’s mandate and mission reflect goals in terms of risk reduction and pursuing public interest; thus, efforts are being pursued to select requirements based on risks. Furthermore, in a continuous effort to progressively move on from the remaining formalism (compliance with formal requirements), processes and tools are being developed. However, priorities in terms of protection of the public welfare are largely concentrated in ‘social-environment conflict’ areas. While this is understandable, other environmental issues in areas where there is no population directly affected, or where the citizens are less well organised, may be neglected – particularly vocal groups may gain strong attention in some cases where environmental risk is actually moderate.

A number of newly introduced indicators (in strategic and operational documents) are based on strategic objectives, and aim at assessing the performance of OEFA in terms of outcomes, but measurements are not yet available. Currently, published documentation by OEFA only covers output indicators.

Choices in terms of planning, priorities, actions, strategy, etc. are for now informed by socio-environmental conflicts and complaints, amongst other criteria. Empirical evidence and data are being progressively collected, in particular through a series of inspection-related activities. Available data is not always being used as much as it could to analyse and quantify harms and risks. Improvements in data and analysis should gradually ensure that the inspection and enforcement framework as well as practices in the environmental sector of Peru evolves into a system fully based on evidence, leaving perception elements behind.

Recommendations

• Collect and increase the use of hard data as evidence, and effectively analysing available data, so that quantitative assessment of harms and risks is taken more into account in both ex ante and ex post regulatory processes.

• The performance of preventive environmental assessment should be multiplied, whenever possible in sites or in establishments located in social-environmental conflict areas. In addition, more specific guidelines need to be developed, so that it is clearer what these assessments are about and how to conduct them.

• OEFA should gradually make sure that it implements a system in which the enforcement and inspections activities in the environmental sector is based on a comprehensive assessment of risks in the complete Peruvian territory. Besides, OEFA should discuss further, in which socio-environmental conflict areas might be one of many focus areas to the extent that the hard evidence dictates.

• When presenting results of OEFA’s activities and performance, there is a need to differentiate the types of measurements – i.e. which are outputs (activities), outcomes (performances) and which data points relate to which indicator.

• As the improvement of data collection and analysis become available, it is crucial to make sure that these actually inform the enforcement and inspections’ policy in the environmental sector. Therefore, other elements such as socio-environmental conflicts and complaints would take a reduced role in setting priorities and planning activities.

References

[6] Defensoría del Pueblo (n.d.), Paz Social y Prevención de Conflictos, https://www.defensoria.gob.pe/areas_tematicas/paz-social-y-prevencion-de-conflictos/ (accessed on 14 October 2019).

[2] OECD (2018), OECD Regulatory Enforcement and Inspections Toolkit, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264303959-en.

[1] OECD (2014), Regulatory Enforcement and Inspections, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264208117-en.

[9] OEFA (2019), Evaluación ex-post de la aplicación de las reglas del servicio de información nacional de Denuncias Ambientales (Sinada). Periodo 2009 - 2017, OEFA, http://www.oefa.gob.pe/?wpfb_dl=34474. (accessed on 14 October 2019).

[4] OEFA (2019), Guía de Orientación para el PLANEFA, http://www.oefa.gob.pe/?wpfb_dl=34241 (accessed on 14 October 2019).

[5] OEFA (2018), Plan Anual de Evaluación y Fiscalización Ambiental del OEFA - PLANEFA 2019, https://www.oefa.gob.pe/planefa/planefa-2019 (accessed on 14 October 2019).

[3] OEFA (n.d.), Lineamientos para la formulación, aprobación, seguimiento y evaluación del cumplimiento del Plan Anual de Evaluación y Fiscalización Ambiental – PLANEFA, https://www.oefa.gob.pe/avisos/lineamientos-para-la-formulacion-aprobacion-y-evaluacion-del-plan-anual-de-evaluacion-y-fiscalizacion-ambiental-planefa (accessed on 14 October 2019).

[11] OEFA (n.d.), Memoria Institucional OEFA 2018, https://www.oefa.gob.pe/publicaciones/memoria-institucional-oefa-2018 (accessed on 14 October 2019).

[10] OEFA (n.d.), Organismo de Evaluación y Fiscalización Ambiental, https://www.gob.pe/minam/oefa (accessed on 14 October 2019).

[7] OEFA (n.d.), Portal de Fiscalización Ambiental, https://publico.oefa.gob.pe/Portalpifa/Intervenciones.do (accessed on 14 October 2019).

[8] OEFA (n.d.), Servicio de Información Nacional de Denuncias Ambientales (SINADA), https://apps.oefa.gob.pe/sinada/ (accessed on 14 October 2019).

Notes

← 1. EFAs are public authorities – at the national, regional and local level – in charge of environmental inspections and enforcement on certain regulated subjects and/or objects.

← 2. Decreto Supremo que aprueba el Texto Único Ordenado de la Ley No. 27444 – Ley del Procedimiento Administrativo General, Supreme decree No. 006-2017-JUS, published on 20 March 2017.