4. The PP and innovation: how well do they mix?
This chapter discusses the intersection of precaution and innovation, and the tension between these two concepts. It explores an imaginary scenario to outline the trade-offs related to innovation and precaution. The chapter also provides a definition of the innovation principle, as well as a selection of examples illustrating how the tension between the PP and the innovation principle may be overcome – at least to a certain extent.
The PP has been criticised for its potential “paralysing effect” on decision-making. This applies especially if the PP is introduced at high-level decision-making where it tends to be used for “yes or no” decisions as opposed to decision-making on the “how” of implementation. According to its critics, it can also be a source of legal uncertainty and stifle innovation.
Proponents of the PP maintain that the PP is vital for innovation — especially at the developmental stage of new technologies — because it provides essential procedures and standards to assess, appraise and control risks. Moreover, some argue that the PP provides a framework that helps to achieve a better balance in public policies and mitigate the difficulties associated with scientific demonstration, prior to justifying preventative measures.
In determining whether precautionary measures against innovations are warranted, the focus needs to be on how, and under which conditions, a given technology should be deployed and applied. The focus should not simply be about whether to allow it or not. In addition, it is important to keep a holistic perspective that encompasses not only the potential hazards associated with an innovation or specific use case, but also considers potential benefits and available alternatives.
Implementation of “innovation-friendly” approaches to precaution and uncertainty are not straightforward. Several attempts have been made in recent years. This includes responsible research and innovation, Safe(r)-by-Design approaches, and frameworks for applying the PP to emerging technologies.
Such implementations are hindered by tensions between the regulatory decisions that scientific findings would seem to warrant on the one hand, and those effectively driven by public perception, stakeholder influence and political considerations, on the other.
Concerning energy technologies, the PP has been invoked in several cases to justify decisions that arguably did not result from evaluative and deliberative processes, but instead were driven by socio-political factors and pre-existing perceptions of risk.
Risk regulation can be key in preventing innovation-induced disasters. The PP should not be seen as a reason to ban or avoid certain innovations altogether, but rather as a means to manage their introduction and deployment in an agile way. To achieve, policymakers and regulators need to ensure that knowledge generated through experimentation and gradual upscaling is regularly and systematically fed into regulatory development and improvement.
With the benefit of hindsight, it is difficult to “decide” whether particular innovations should have been introduced or allowed – and maybe this difficulty can help clarify why an “all or nothing” approach may not be optimal in most situations.
For illustrative purposes, a scenario could be conceived, in which a modern regulator had to approve the first application of the steam engine, either for a railway or textile spinning or weaving. What would have been the correct PP-driven decision, if this regulator had access to modern scientific knowledge? Arguably, this imaginary regulator could very well have considered the immense downside risk of global warming to be unacceptable, regardless of the potential upsides of the technology. In retrospect, the energy revolution of the late 18th and early 19th centuries brought about industrialisation and further technological progress, which not only brought mankind to face the chasm of climate disaster, but also made possible modern warfare, as well as destruction of biodiversity on a greater scale and speed than ever before.
However, this would certainly be a very partial view of the impact of these (then new) technologies. Indeed, the energy revolution also brought about unprecedented economic and social development, and generally immensely improved material welfare, as humanity exited its previous condition of prevailing scarcity. This, in turn, contributed to transformations in political and social systems that brought about, for many countries at least, major improvements in personal freedoms and human rights.1 Of course, even someone in the early 1800s who was trying to imagine the future systems and scientific knowledge would have had no way to predict the exact scenarios unlocked by use of these new technologies, let alone model their relative probabilities. What would, thus, have been an appropriate PP-informed decision?
Rather than an “all or nothing” decision, the imaginary regulator could have taken a variety of measures, such as authorising the use of the technology while mandating the periodic review of its impacts on health and the environment. Or, the regulator could run a series of small-scale pilots or regulatory experiments to gain a deeper understanding of the harmful effects of the technology and how they could be better managed and mitigated. Crucial elements in this respect would include: a) apprehending the PP and its realm of application in terms of systems and understanding the relevant impacts and interdependencies; b) monitoring closely what is changing/known/unknown; active investigation and testing, especially with long term assessments; c) communicating and reframing as appropriate, including a readiness to modify regulatory approaches when the signals change ( for example, those available through pharmacovigilance, environmental change, climate data, etc).
As a matter of fact, risk regulation can be key in preventing innovation-induced disaster. From this perspective, the PP should not be seen as a reason to ban or avoid certain innovations altogether, but rather as a means to manage their introduction and deployment in an agile way. The PP can achieve this by ensuring that the knowledge generated through experimentation and gradual upscaling is regularly and systematically fed into regulatory development and improvement.
Balancing precautionary approaches and innovation is challenging. A fundamental question is that of characterising “appropriate use” of the PP as far as its articulation with innovation is concerned. Indeed, in recent years, the debate around the PP has intensified concomitantly with the articulation of the innovation principle. Difficult questions have been raised: is the precautionary principle an obstacle to progress and innovation? How can the PP and the innovation principle reinforce each other?
For some, the PP is a source of legal uncertainty and may also undermine progress and innovation by not factoring in the opportunity costs of forgone progress (Orset, 2014[1]) (Institut économique Molinari, 2013[2]). Sunstein has referred to the potential “paralysing effect” of measures underpinned by the precautionary principle (Sunstein, 2002[3]), which can be argued to have implications for risk-taking and innovation. Similarly, the UK Institute of Economic Affairs (IEA) has voiced “concerns that badly designed and targeted regulation holds back innovation, due in part to excessive restrictions brought about by misapplication of the precautionary principle”. The IEA has called for a more proportionate application of the PP “that gives due weight to the benefits to (amongst other things) the environment, human health and wellbeing that innovation and economic growth can bring” (Hewson, 2021[4]). Moreover, the European Risk Forum (ERF) argues that regulation focused solely on avoiding risk and removing scientific uncertainty stifles technological innovation; it suggests the consideration another principle, the innovation principle to ensure the systematic consideration of policy and regulation’s impact on innovation.
Conversely, others such as the European Political Strategy Centre (EPSC) maintain that the PP is vital to innovation, especially at the developmental stage of new technologies. This is because it provides essential procedures and standards to assess, appraise and control risks; an approach in stark contrast to the “move fast and break things” philosophy advocated by some industry players. In addition, it has been argued that the framework of the PP supports a better balance in public policies [public policy debate?] and helps to mitigate difficulties associated with scientific demonstration prior to justifying preventative measures (European Parliamentary Research Service, 2015[5]). In a similar vein, according to the European Environment Agency (EEA), appropriate use of the PP can promote a wide range of technologies and activities (e.g. regarding the development of agroecology solutions).
The innovation principle was first proposed for adoption in risk management and regulatory practice in 2013 by the ERF in an open letter to the then three Presidents of the EU institutions: “Whenever policy or regulatory decisions are under consideration the impact on innovation as a driver for jobs and growth should be assessed and addressed”. In contrast to the precautionary principle, the innovation “principle” has no legal anchoring. Instead, at EU level, the innovation principle is considered a regulatory approach that contributes to achieving EU policy objectives (e.g. jobs and growth, the green and digital transformations, competitiveness and overall value creation) by ensuring that regulation creates the best possible ecosystem for (beneficial) innovation.
According to a 2017 paper, despite its lack of legal anchoring, the innovation principle has been gaining ground and is being referred to in many policy documents. The RECIPES study has identified several initiatives that give the innovation principle a role in law. At EU-level, this includes the European Commission’s (DG R&I) definition of the innovation principle as a “tool to help achieve EU policy objectives by ensuring that legislation is designed in a way that creates the best possible conditions for innovation to flourish”.
The OECD Recommendation of the Council for Agile Regulatory Governance to Harness Innovation also acknowledges regulation’s role in helping to ensure that innovation serves to promote sustainable and inclusive economic development and address global challenges. It provides a conceptual framework and guidance to help governments realise the full potential of innovation in high-uncertainty contexts by enhancing its benefits for societies while addressing its risks. Similarly, the OECD Declaration of Public Sector Innovation – which has been adhered to by 43 countries since 2019 recognise the benefits that can come from enabling experimentation in core systems (such as the use of digital technologies, budgeting, risk management and reporting) and explore whether and how they can be achieved.
Source: (European Risk Forum, 2015[6]); (Vos and De Smedt, 2020[7]); (OECD, 2021[8]); (European Commission, DG R&I, 2022[9])
Going forward, acknowledgement that both the precaution and innovation principles are not “self-evident” when it comes to their application will be key. Both principles can provide guidance and draw attention to aspects and issues that may otherwise be undervalued or overlooked. However, there will always be trade-offs between risks, as well as the nature and distribution of benefits — all of which must be assessed and prioritised. There is also a need to find more agile solutions that go beyond the “ban or allow” dichotomy.
According to Garnett, Van Calster and Reins, a possible future challenge for the EU is how the innovation principle — if it becomes more established — can be linked to and interrelate with the PP. They conclude that a qualified innovation principle would be perfectly compatible with both the EU’s Better Regulation objectives and the PP, as well as with the EU’s circular economy goals and commitment to responsible research and innovation. Such a qualified innovation principle would need to recognise the need to act responsibly, while encouraging reasonable risk-taking in a competitive global market (Garnett, K., Van Calster, G. and Reins, L., 2018[10]).
The European Political Strategy Centre (EPSC) highlights that “the precautionary principle is of particular importance for innovation because, especially at an early stage of a new technique or approach, the possibility of a risk often cannot be ruled out. It provides procedures and criteria to assess, appraise and manage risks. An integral part of risk management, as envisaged by the PP, is the examination of the potential benefits and costs of action, or lack of action” (European Political Strategy Centre, 2016[11]).
It has been argued that the PP and innovation principle may at some point “collide” should the latter be incorporated into EU law (Garnett, K., Van Calster, G. and Reins, L., 2018[10]). However, it is also argued that there are ways to reconcile the innovation principle and the PP. One of these is to consider the former as a complement to the latter. According to this view, regulatory assessment processes should try to reconcile the two principles and achieve a more balanced use of the PP. However, this view does not answer the question of what exactly should be balanced with the PP, and how this balancing could be carried out in practice. A second way to reconcile the two involves weighting systematically precautionary measures against the societal benefits of innovations. This approach seeks to develop a science-based appraisal process in line with the agendas of Better Regulation and responsible research and innovation (Vos and De Smedt, 2020[7]).
Responsible research and innovation (RRI)
The dimensions of anticipation and inclusion are central to the responsible innovation concept. Responsible research and innovation (RRI) has been strongly promoted by European Commission as an innovative governance concept, notably in its 2014-2020 research and the Horizon 2020 innovation programme. RRI seeks to reconcile innovation with sufficient safeguard levels. Four elements characterise the approach, which is included in the proposed guidance on the application of the PP as developed under the EU RECIPES project (RECIPES, 2021[12]), which includes four elements:
Anticipation: “Involves systematic thinking aimed at increasing resilience, while revealing new opportunities for innovation and the shaping of agendas for socially-robust risk research.”
Reflexivity: “At the level of institutional practice, means holding a mirror up to one’s own activities, commitments and assumptions, being aware of the limits of knowledge and being mindful that a particular framing of an issue may not be universally held”.
Inclusion: taking the time to involve different stakeholders in order to expose and explore the impacts of a new technology can have on different communities.
Responsiveness: “Responsible innovation requires a capacity to change shape or direction in response to stakeholder and public values and changing circumstances” (Stilgoe et al., 2013[13]).
The EU’s research and innovation framework programme highlights the necessity to foster social and environmental responsibility and ethics in the governance of science and technology. This is realised by developing activities that: 1) respond to significant societal needs and challenges; 2) involve a range of stakeholders for the purpose of mutual learning, and 3) anticipate potential problems, identify alternatives and study the underlying values.
A sub-section later in this chapter is dedicated to Safer by design and regulatory preparedness and provides further insights into a specific elaboration of RRI.
Precaution and sustainable development in the EU’s Environment Action Programme
The European Environment Agency (EEA) also supports the coexistence of the two principles by stating that “precautionary actions can be seen to stimulate rather than hinder innovation”. At the same time, it observes that the speed and scale of today's technological innovations can inhibit timely action because, by the time clear evidence of harm has been established, the technology has been modified, thereby allowing claims of safety to be subsequently re-asserted.
According to the EEA, these features of current technological innovation strengthen the case for taking early warning signals more seriously and acting on lower strengths of evidence than those normally used to conclude to “scientific causality”. The EEA states that most of the historical case studies show that by the time such strong evidence of causality becomes available, “the harm to people and ecosystems has become more diverse and widespread than when first identified, and may even have been caused by much lower exposures than those initially considered dangerous” (European Environment Agency, 2013[14]).
The EU’s 7th Environment Action Programme (EAP), in turn, stresses that — with new technologies — comes the need to better understand, assess and manage the potential environmental and human health risks that they bring. It suggests that major technological innovations should be accompanied by public dialogue and participatory processes, with need for a broad debate about the possible trade-offs that are deemed acceptable considering the sometimes incomplete or uncertain information about emerging risks. (Official Journal of the European Union, 2013[15])
On a related note, a report by the UK Interdepartmental Liaison Group on Risk Assessment (UK-ILGRA) suggests that the precautionary position adopted "should reflect the commitment to sustainable development that gives full weight to economic, social and environmental factors." It adds that the principle should not obstruct innovation and that, applied properly, it is a positive policy tool "to encourage technological innovation and sustainable development by helping to engender stakeholder confidence that appropriate risk control measures are in place" (Interdepartmental Liaison Group on Risk Assessment, 2002[16]).
The following section discusses the application of the PP to emerging technologies. It explores in further detail the concept of ‘safer by design’ and regulatory preparedness, as well as the EU Commission’s approach to regulating artificial intelligence. The section also looks into a series of case studies related to the RECIPE project (REconciling sCience, Innovation and Precaution through the Engagement of Stakeholders), which analyse the application of the precautionary principle across different thematic areas.
In a 2016 article, several leading authors contended that precaution can be consistent with support of science and innovation. The article noted that “the charge that precaution means giving up on technology depends both on the triggers and the precautionary recommendations”. Quoting a report on gene drive research by the US National Academies of Science, Engineering, and Medicine (NASEM), it called for “targeted but meaningful measures” to “encourage a broader range of perspectives on — and questions about — the technology”. The article also advocated for a “robust and iterative assessment that can incrementally reduce uncertainty surrounding its outcomes and probabilities” and “ensure that risks are acceptable to the relevant publics and are reduced to the greatest extent possible” (Kaebnick,G. E. et al, 2016[17]).
Regulatory approaches (including precautionary) related to innovation and emerging technologies have also been analysed by Stirling, who claims that “Precaution is about steering innovation, not blocking it” (Stirling, 2016[18]). The author proposes a framework for applying the PP to emerging technologies. Figure 4.1 provides an overview of this framework.
The general framework shown above encompasses 17 key considerations: what needs to be demonstrated in a precautionary technology appraisal, with appraisal criteria that includes independence from vested interests, and the examination of uncertainties, sensitivities and possible scenarios. An initial screening process for targeted precautionary appraisal (including socio-political factors) is also foreseen as an aid to determining when a precautionary appraisal, stakeholder deliberation or risk assessment are needed. The various stages of the framework are conceived as interlinked elements, all of which “feed into the communication, evaluation and management of identified threats” (European Commission, 2017[19]) (Stirling, 2016[18]).
As always, the challenge lies in the implementation of this approach. There can be material tensions between the regulatory decisions that scientific findings would seem to warrant on the one hand, and those effectively driven by public perception, stakeholder influence, and political considerations, on the other. As far as energy technologies are concerned, the PP has arguably been invoked in a number of cases to justify decisions that did not seem to result from the kind of evaluative and deliberative processes put forward by Stirling among others. As discussed earlier in this report, it is not unusual for policy and regulatory decisions to be driven by factors other than scientific knowledge and evidence, such as socio-political factors and perceptions of risk.
Source: (Stirling, 2016[18]).
Decisions around the EU taxonomy of environmentally sustainable activities illustrate well the complexity of such decision-making processes. After nuclear energy was initially excluded, in 2020, the European Commission launched in-depth work to assess whether to include nuclear energy in in this taxonomy and entrusted its Joint Research Centre (JRC) with drafting a technical report on the “do no significant harm” aspects of nuclear energy. This report’s findings notably indicate that “nuclear energy-based electricity generation can be considered as an activity significantly contributing to the climate change mitigation objective”, and that “all potentially harmful impacts of the various nuclear energy lifecycle phases on human health and the environment can be duly prevented or avoided” (European Commission Joint Research Centre, 2021[20]). To best support the discussion, the JRC brought together several group of experts2. Nonetheless, policy discussions around nuclear energy continue to often focus on potential consequences of an extreme accident, rather than on the aggregate average risk. This may reflect both other policy preferences and deep divergences in how we perceive risk, which often do not match quantitative risk assessments (Slovic, 1987[21]) (Slovic and Peters, 2006[22]). Such discussions and decisions may end up focusing on one aspect (consequences of a possible tail-risk accident), at the expense of a more holistic consideration of the risks and benefits (including climate, biodiversity, etc.) of different options, where any decision is about risk trade-offs rather than the total avoidance of risk, which is impossible in the climate and energy context. In practice, not using any low-carbon energy source often means using a higher-carbon alternative, which is itself not without other safety and environmental risks, and thus this report emphasizes the importance to understand precaution as a careful consideration of this balance of aggregate risks and benefits, with due importance being given to the climate consequences given the existing emergency.
Safer by design and regulatory preparedness
Authors such as Kletz (Wolke, 1985[23]) and Van Gelder et al. have made substantive research contributions to the analysis of Safe(r)-by-Design approaches (SbD, also named Safe-by-Design or Safety-by-Design). Van Gelder et al. present SdB as a tool that can help shape governance arrangements for accommodating and incentivising safety, while fully acknowledging uncertainty by organising for responsibility through monitoring practices and facilitating adaptive change (van Gelder et al., 2021[24]).
After assessing SbD’s application in various engineering disciplines (including construction engineering, chemical engineering, aerospace engineering, urban engineering, software engineering, bioengineering, nano-engineering, and cyber space engineering), the authors conclude that:
Safe-by-Design is best considered as a specific elaboration of Responsible Research and Innovation, with an explicit focus on safety in relation to other important values in engineering such as well-being, sustainability, equity, and affordability. Safe-by-Design provides for an intellectual venue where social science and the humanities (SSH) collaborate on technological developments and innovation by helping to proactively incorporate safety considerations into engineering practices, while navigating between the extremes of technological optimism and disproportionate precaution (van Gelder et al., 2021[24]).
The authors also identify the need for further research and analysis, and go on to formulate qualitative principles for inherently safer design (e.g. green chemistry). In their conclusions, they highlight the importance of having a context-specific understanding of “disciplinary and regulatory histories”. This supports the alignment of safety practices with other values “at the heart of societal challenges, from climate mitigation to building resilient societies capable of dealing with the pressures of a global pandemic”.
They also conclude that it would be valuable for developers and regulators to “work together to develop, test, and assess how different safety-oriented design approaches and dedicated governance arrangements for warranting safety and security fare in different contexts and to investigate how best to adapt such approaches in response to both lessons learned and evolved circumstances” (van Gelder et al., 2021[24]).
The OECD has also developed conceptual and analytical work on this topic. Its report “Moving Towards a Safe(r) Innovation Approach (SIA) for More Sustainable Nanomaterials and Nano-enabled Products” proposes a combination of Safe(r)-by-Design and regulatory preparedness. This combined process can raise awareness and improve decision-making effectiveness, leading to a Safe(r) Innovation Approach for nanomaterials and nano-enabled products (OECD, 2020[25]).
According to the report, SbD refers to identifying the risks and uncertainties that concern humans and the environment at an early phase of the innovation process. This is with the goal of minimising uncertainties, potential hazard(s) and/or exposure. The SbD approach addresses the safety of the material/product and associated processes through its whole life cycle: from the research and development phase to production, use, recycling and disposal. Regulatory preparedness, in turn, refers to the capacity of regulators (including policymakers), to anticipate the regulatory challenges posed by emerging technologies such as nanotechnology — particularly those relating to human and environmental safety. This requires that regulators become aware of, and understand, innovations sufficiently early in the process to take appropriate action, and that appropriate regulatory tools are modified or developed as needed. Regulatory preparedness helps to ensure that innovative materials and products undergo a suitable (and if appropriate, adapted) safety assessment before entering the market. Regulatory preparedness requires dialogue and knowledge-sharing among regulators, and between regulators and innovators, industry actors and other stakeholders.
The SbD approach has very relevant applications in facilitating the management of risks, including imperfectly quantifiable risks, to support the energy transition. This is the case for instance with electrolysers for the production of hydrogen. The SbD approach can be applied to the construction and operation of hydrogen electrolysers. Electrolyser manufacturers should design their products in accordance with applicable standards. For example, ISO 22734:2019 is an International standard that defines the construction, safety, and performance requirements of hydrogen electrolysers and can be used for certification purposes. This standard applies for industrial and commercial use. One point to consider, though, is that the rapid growth of hydrogen technologies and the decarbonisation goals that countries have set demand large-scale installations, where the operating experience is limited. Hence, the need to develop safety best practices and guidelines for large-scale electrolysis plants is emerged.
For a more elaborate discussion on regulatory preparedness, the reader may usefully refer to the report summarising the presentations and discussions at the first NanoReg2 Workshop on Regulatory Preparedness for Innovation in Nanotechnology (Publications Office of the European Union, 2018[26]).
The role of good communication in SbD and regulatory preparedness
Related work that notably builds on concrete project experience includes the development of a methodological SbD approach. One of its important conclusions is that a continuous and proactive combination of interconnected activities is required for ensuring regulatory preparedness. As a result, anticipation (e.g. horizon scanning), was seen as particularly important, as was communication between regulators, innovators (industry) and other stakeholders.
Regulators need to become aware of innovative products under development to ensure that the legislation and methods for safety assessment are available and adequate. Similarly, innovators must be aware of regulatory requirements and their likely evolution. This mutual awareness helps to develop safe products and to avoid delays or other problems in obtaining market approval.
Awareness can be achieved through communication and this requires trust. A format for achieving this includes the promotion of "trusted environments" for confidential inquiries and information sharing. Such trusted environments are physical or virtual spaces in which industry actors, innovators, governmental institutions and other stakeholders can share and exchange knowledge, information, and views on new technologies. Furthermore, regulators need early access to the existing information and data relevant to the safety assessment of innovative products. This enables them to provide timely guidance and advice to industry, as well as develop their own strategies for dealing with uncertainty (e.g. by applying the PP).
For the PP to be applied in a proactive and time-efficient way, regulators need to acquire early any information about a new technology — while it is still being developed. With early knowledge of a technology’s characteristics and potential safety concerns, the necessary regulatory tools — such as adapted legislation and appropriate safety assessment methodologies — can already be in place by the time industry is ready to seek any necessary market approval.
The participation of a variety of stakeholders has also been found to be crucial in implementing regulatory preparedness. This can help regulators to anticipate the need for new or modified regulatory tools, whilst reducing for innovators and industry those uncertainties associated with future safety legislation and the regulation applicable to emerging technologies.
Challenges of the SbD approach
There are several issues that can be obstacles to the implementation of the SbD approach:
Cost and resource constraints (additional costs for technical and human resources; extra time in implementation of SbD). A number of incentives have been proposed to resolve this: allocate resources with particular focus on the benefits of SbD; develop structures to help design products and support usages at a reduced price; provide tax reduction of nanomaterial SbD; specific funding to support regulators in making transition to this approach.
Lack of knowledge/data gaps (in identification of potential issues/problems; lack of specific knowledge at early stages; lack of structure for training; education and information). Incentives proposed to resolve this: publish clear and concise guidance; indicate area and cases where SbD is supported; offer training and access to infrastructures; greater involvement of SMEs.
Cultural change (innovator/manufacturer have poor understanding of their responsibilities in SbD). Incentives proposed to resolve this: establish an international agreement on the SbD concept with support of national governments; create international certificates or rewards for applying SbD; create governmental platforms to provide technical support on SbD to SMEs; educate the public on the benefits brought by SbD (for example by promoting success stories).
Lack of frameworks, guidance and tools (gaps between what is needed and what is available, lack of predictive tools in risk assessment, life cycle analysis and socio-economic analysis). Incentives proposed to resolve this: need of a supportive regulatory environment; development of normative activities; creation of certifications; introduction of SbD to the curricula of techno scientific studies.
Inadequate regulation (lack of regulatory process to support SbD for nanomaterials (including legal instruments and liabilities), lack of a discussion platform between regulators and innovators). Incentives proposed to resolve this: develop regulatory requirements and corresponding measures for fulfilment of the essential social and environmental aspects of SbD.
Insufficient communication, collaboration and open-mindedness (competitive industrial environment may make communication difficult between industry actors and regulators/policymakers). Incentives proposed to resolve this: develop an International Consensus on SbD at a high level; improve communication between innovators and regulators (e.g. by establishing open channels or platforms of communication and interaction between them); greater promotion in public and across the media of SbD and its success stories.
The European Commission’s risk-based approach to regulating Artificial Intelligence (AI)
Identifying safety and security risks caused by AI is an essential task for the EU and its Member States as they aim to build safe AI in order to use the full potential of this technology while mitigating its potential negative impacts (van der Linden-Smith, M., Dufour, R., Smits, J., & Koehof, J., 2021[27]). In its 2021 Draft Regulation for Artificial Intelligence, the European Commission proposed to establish a technology-neutral definition of AI systems in EU law and to lay down a classification for AI systems with different requirements and obligations following a risk-based approach. The proposed classification is as follows:
Unacceptable-risk AI systems include: “(1) subliminal, manipulative, or exploitative systems that cause harm, (2) real-time, remote biometric identification systems used in public spaces for law enforcement, and (3) all forms of social scoring, such as AI or technology that evaluates an individual’s trustworthiness based on social behaviour or predicted personality traits.”
High-risk AI systems include those that evaluate consumer creditworthiness, assist with recruiting or managing employees, or use biometric identification, as well as others that are less relevant to business organisations. Under the proposed regulation, the EU would review and potentially update the list of systems included in this category on an annual basis.
Limited- and minimal-risk AI systems include many of the AI applications currently used throughout the business world, such as AI chatbots and AI-powered inventory management (European Commission, 2021[28]).
AI systems in the unacceptable-risk category are prohibited. As currently proposed, high-risk systems would be subject to the largest set of requirements, including human oversight and checks on transparency, cybersecurity, data quality, monitoring, as well as reporting obligations. Lastly, the limited and minimal risk AI systems merely have an obligation of transparency. To classify the AI, the Commission examines features such as opacity, complexity, dependency on data, autonomous behaviour, which can adversely affect several fundamental rights and users’ safety.
The following section summarises the findings of several case studies on the application of the PP to a range of areas involving the use of (disruptive) technology.
Case study analysis (RECIPES project)
The RECIPES project (REconciling sCience, Innovation and Precaution through the Engagement of Stakeholders) was set up in 2019 by the European Commission and is based on the idea that the application of the PP and responsible innovation do not necessarily contradict each other. The project analyses the application of the precautionary principle across different areas and presents a series of thematic case studies. Box 4.2 presents selected summaries of this analysis and puts forward several additional elements.
The information presented in this subsection highlights the difficulties associated with determining whether precautionary measures are warranted in the presence of certain technologies and innovations. Particularly challenging is the fact that pre-existing perceptions are likely to determine the course of action to be pursued. Several of the examples above do show, however, that the relevant question is often about how and under which conditions a given technology should be deployed and applied, rather than simply about allowing it or not. In addition, it is important to keep a holistic perspective that encompasses not only the potential hazard associated with an innovation or specific use case, but also related potential benefits and available alternatives – including any risk-risk trade-offs.
Gene drives
This case study investigates the application of the PP to gene drives, an area linked to many benefits including sustained and durable genetic change. There exists, however, a high degree of epistemic uncertainty surrounding the impact of gene drives (the alteration of a population may bring unforeseen and unknowable consequences). The case study illustrates how — despite differences in view about how the PP should be implemented — there is a consensus amongst gene drive researchers, policymakers and other stakeholders that the application of the PP is important in the context of gene drives.
GMOs
The case study highlights remaining uncertainty about the consequences of using GMOs as food and food ingredients. It argues that it is not entirely possible to determine the extent and likelihood of possible harms — something which strengthens the scientific uncertainty — and concludes with the need for a precautionary approach. As discussed earlier in this report, there seems to be, however, scant scientific evidence of significant hazards directly connected with the use of GE crops (Nicolia et al., 2013[29]).
Endocrine disruptors (EDCs)
According to this case study, “the precautionary principle is of utmost relevance for the governance of EDCs”. There is scientific uncertainty about the effects of EDCs because it is difficult to determine the precise causal chains through which EDCs act on the hormonal system of both humans and wildlife. The growing scientific evidence on the negative effects of EDCs for health and the environment caused debate within the EU about the identification and regulation of EDCs. From 2014 to 2017, there was controversy surrounding the Commission’s delay to set out scientific criteria for the determination of endocrine disrupting properties. Despite the subsequent establishment of criteria for the identification of EDCs in several – but not all – EU legislative areas, there remain ongoing concerns about the suitability of the EU’s regulatory framework to EDCs.
In stark contrast with GMOs and nuclear energy, the benefits of EDCs —or the cost of parting with them — are limited and suitable alternatives exist (even if they are not without their own risks).
Nanotechnology
According to this case study, nanotechnology and emerging technologies are defined by uncertainties rather than risks. This is due notably to a lack of data and analytical methods, lack of long-term experience, scientific ambiguities, and terminological vagueness, etc. The European Commission produced an “integrated and responsible approach” on nanotechnology in its Nanoscience and Nanotechnology Action Plan of 2005. This was largely based on the PP. Consequently, Member States (e.g. Germany, Austria, and Switzerland) have been implementing regulatory measures such as safety research programmes, standardisation projects or the establishment of evaluation processes and commissions to integrate safety issues into innovation at an early stage.
Glyphosate
The PP effectively shapes the approval procedure and regulation of pesticides in the EU. However, as in the risk assessment of glyphosate at EU level no risk was determined, no precautionary measure in the form of a ban was taken. This reflected the fact that glyphosate was assessed as presenting no clear and demonstrated health risk, while any environmental harm should be cautiously weighed against the fact that available alternatives to glyphosate (extra tilling, other compounds) are not necessarily without negative environmental impacts, some of which are possibly worse than the substance they are meant to replace. The assessment and the decision have been contested by various stakeholders, and some Member States have introduced partial or complete glyphosate bans at national level. This example shows that “precaution” or the PP itself are not “cookie cutter” approaches that automatically deliver a clear and one-sided conclusion, or always lead to a more restrictive outcome. Indeed, in this case, the decision was taken not to act (as alternatives appeared rather worse), at least at EU level.
Urban wastewater
This case study deals with the implementation of the EU Urban Wastewater Treatment Directive 91/271/EC (UWWTD). The UWWTD is part of the European Water Framework Directive (WFD). It refers directly to the PP, the control at source principle, and the polluter pays principle (Art. 191(2) of the Treaty on the Functioning of the European Union. This case study investigates the dynamics that occur under an established precautionary principle regulatory regime. The challenges posed by this regime instigate innovative practices and shift risks (e.g. financial risk) to other sectors. The case study shows that if the precautionary principle is taken seriously (in the sense of Responsible Research and Innovation), then the accompanying financial risks can also be brought under control.
Artificial intelligence in healthcare
This case study indicates that the PP may be applicable to the use of clinical decision support systems, but only in specific circumstances. A variety of scientific research suggests that human rights and public health are at issue in some uses of Clinical Decision Support Systems (CDSS). Because of the difficulty in many cases of defining the specific outcome and statistical probability of the risks, the PP is, according to the case study, more suitable than the principle of prevention. From this perspective, the PP may enable appropriate regulatory and technical boundary setting for CDSS — for instance, by limiting the medical procedures in which a CDSS can be used, or the amount of human oversight that is necessary for important decisions in healthcare. Arguably, however, the crucial question in this context would probably be about identifying which decision-making system (be it AI-supported or not) will be less-biased and more evidence-based and, ultimately, more effective.
Microplastics in food products and cosmetics
The European Commission has been working on rules to prohibit microplastics in certain products. This will be done via the REACH regulation, which is based on the precautionary principle, to protect the environment. According to the case study, this prohibition will not include an end date and will not be changed even if new scientific evidence emerges”. This is because the intentional release of microplastics into the environment is now seen as unacceptable. Moreover, the use of microplastics carries few to zero benefits, while generating very substantial harm.
Source: RECIPES, D2.4.1 Intra Case Study Analysis (2020).
References
[30] Engels, F. (1845), “The Condition of the Working Class in England”, Marx/Engels Internet Archive, https://www.marxists.org/archive/marx/works/1845/condition-working-class/index.htm.
[31] European Commission (2022), EU taxonomy for sustainable activities, https://ec.europa.eu/info/business-economy-euro/banking-and-finance/sustainable-finance/eu-taxonomy-sustainable-activities_en.
[28] European Commission (2021), Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts COM/2021/206 final..
[19] European Commission (2017), Future brief: The precautionary principle: decision-making under uncertainty.
[20] European Commission Joint Research Centre (2021), Technical assessment of nuclear energy with respect to the ‘do no significant harm’ criteria of Regulation (EU) 2020/852 (‘Taxonomy Regulation’), https://ec.europa.eu/info/sites/default/files/business_economy_euro/banking_and_finance/documents/210329-jrc-report-nuclear-energy-assessment_en.pdf.
[9] European Commission, DG R&I (2022), The Innovation Principle, https://research-and-innovation.ec.europa.eu/system/files/2022-07/ec_rtd_factsheet-innovation-principle.pdf.
[14] European Environment Agency (2013), Late lessons from early warnings: science, precaution, innovation, https://www.eea.europa.eu/publications/late-lessons-2/late-lessons-2-full-report/late-lessons-from-early-warnings/view.
[5] European Parliamentary Research Service (2015), The Precautionary Principle. Definitions, applications and governance, https://www.europarl.europa.eu/RegData/etudes/IDAN/2015/573876/EPRS_IDA(2015)573876_EN.pdf.
[11] European Political Strategy Centre (2016), Towards an Innovation Principle Endorsed by Better Regulation.
[6] European Risk Forum (2015), Better Framework for Innovation: Fuelling EU policies with an Innovation Principle.
[10] Garnett, K., Van Calster, G. and Reins, L. (2018), Towards an innovation principle: an industry trump or shortening the odds on environmental protection?, https://www.tandfonline.com/doi/full/10.1080/17579961.2018.1455023.
[4] Hewson, V. (2021), A British Innovation Principle, https://iea.org.uk/publications/a-british-innovation-principle/.
[2] Institut économique Molinari (2013), Bilan du principe de précaution : des effets pervers sous-estimés, selon une nouvelle étude de l’Institut économique Molinari (IEM), https://www.institutmolinari.org/2013/09/30/bilan-du-principe-de-precaution-des-effets-pervers-sous-estimes-selon-une-nouvelle-etude-de-linstitut-economique-molinari-iem/.
[16] Interdepartmental Liaison Group on Risk Assessment (2002), The Precautionary Principle: Policy and Application.
[17] Kaebnick,G. E. et al (2016), “Precaution and governance”, Science, https://research.ncsu.edu/ges/files/2014/03/Kaebnick-et-al-2016-Precaution-and-governance-of-emerging-technologies.pdf.
[29] Nicolia, A. et al. (2013), “An overview of the last 10 years of genetically engineered crop safety research”, Critical Reviews in Biotechnology, Vol. 34/1, pp. 77-88, https://doi.org/10.3109/07388551.2013.823595.
[8] OECD (2021), Recommendation of the Council for Agile Regulatory Governance to Harness Innovation. OECD/LEGAL/0464.
[25] OECD (2020), Moving Towards a Safe(r) Innovation Approach (SIA) for More Sustainable Nanomaterials and Nano-enabled Products, https://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=env/jm/mono(2020)36/REV1&doclanguage=en.
[15] Official Journal of the European Union (2013), “Decision No 1386/2013/EU of the European Parliament and of the Council on a General Union Environment Action Programme to 2020 ‘Living well, within the limits of our planet’”, https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013D1386.
[1] Orset, C. (2014), Innovation and The Precautionary Principle. Economics of Innovation and New Technology, Taylor & Francis (Routledge).
[26] Publications Office of the European Union (2018), Workshop on Regulatory Preparedness for Innovation in Nanotechnology, https://publications.jrc.ec.europa.eu/repository/handle/JRC112766.
[12] RECIPES (2021), Guidance on the application of the precautionary principle in the EU.
[21] Slovic, P. (1987), “Perception of Risk”, Science, Vol. 236/4799, pp. 280-285, https://doi.org/10.1126/science.3563507.
[22] Slovic, P. and E. Peters (2006), “Risk Perception and Affect”, Current Directions in Psychological Science, Vol. 15/6, pp. 322-325, https://doi.org/10.1111/j.1467-8721.2006.00461.x.
[13] Stilgoe et al. (2013), Developing a framework for responsible innovation..
[18] Stirling, A. (2016), Precaution in the Governance of Technology. SPRU Working Paper Series, https://www.researchgate.net/publication/305986930_Precaution_in_the_Governance_of_Technology_SWPS_2016-14/link/57a874da08ae0107eee467f3/download.
[3] Sunstein, C. (2002), Beyond the Precautionary Principle.
[27] van der Linden-Smith, M., Dufour, R., Smits, J., & Koehof, J. (2021), AI or More? A Risk-based Approach to a Technology-based Society. Oxford Business Law Blog..
[24] van Gelder, P. et al. (2021), “Safe-by-Design in Engineering: An Overview and Comparative Analysis of Engineering Disciplines”, International Journal of Environmental Research and Public Health, Vol. 18/12, p. 6329, https://doi.org/10.3390/ijerph18126329.
[7] Vos, E. and K. De Smedt (2020), RECIPES, WP1 Report: Taking stock as a basis for the effect of the precautionary principle since 2000, https://recipes-project.eu/sites/default/files/2021-01/Report_Taking%20stock%20as%20a%20basis%20for%20the%20effect%20of%20the%20precautionary%20principle%20since%202000_Final.pdf.
[23] Wolke, H. (1985), “Cheaper, safer plants or wealth and safety at work - notes on inherently safer and simpler plants. Loos Prevention, Rugby/England: The Institution of Chemical Engineers, 1984”, Acta Biotechnologica, Vol. 5/3, pp. 262-262, https://doi.org/10.1002/abio.370050310.
Notes
← 1. It should, of course, be noted that this process did involve extended periods of deepened labour, health and social inequalities (with significant deteriorations from even basic agrarian conditions). For example, in The Condition of the Working Class in England Condition, Engels argued that the Industrial Revolution had made workers worse off in terms of health and living and working conditions; e.g. referring to “excessive mortality, an unbroken series of epidemics, [and] a progressive deterioration in the physique of the working population” (Engels, 1845[30]).
← 2. The JRC report was subsequently reviewed by two sets of experts: the Group of Experts on radiation protection and waste management under Article 31 of the Euratom Treaty, and the Scientific Committee on Health, Environmental and Emerging Risks on environmental impacts (European Commission, 2022[31]). These reviews served to qualify a number of these findings.