Morrison Foerster is an international law firm with lawyers practicing throughout the United States, Asia, and Europe. Click to discover more.

mofo_share_image.jpg

Morrison Foerster

favicon.ico

favicon-16x16.png

favicon-32x32.png

apple-touch-icon.png

MoFo Favicons

Terms / Notices

Cookies

Privacy Policy

Attorney Advertising

Secure Login

Alumni

  Facebook

MoFo Facebook

  Linkedin

MoFo LinkedIN

Twitter

MoFo Twitter

YouTube

MoFo Youtube

Morrison Foerster - Footer

You have not solved the recaptcha challenge yet or session expired, try again.

Disclaimer

<p>Unsolicited e-mails and information sent to Morrison & Foerster will not be considered confidential, may be disclosed to others pursuant to our <a href="https://www.mofo.com/about/privacy-policy.html"><u><em>Privacy Policy</em></u></a>, may not receive a response, and do not create an attorney-client relationship with Morrison & Foerster. If you are not already a client of Morrison & Foerster, do not include any confidential information in this message. Also, please note that our attorneys do not seek to practice law in any jurisdiction in which they are not properly authorized to do so.</p>

Feedback

mofo-logo-2022.svg

People

Capabilities

Resources

About

Culture

Offices

Careers

Stay Connected

The draft EU Regulation on Artificial Intelligence imposes a broad range of requirements on both the public and private sectors, which are summarized in this alert. Some of these requirements already apply (in a similar form) under the EU General Data Protection Regulation (GDPR). This begs the question: What is the impact of the Regulation on the privacy sector, and what requirements already apply?

Privacy and the EU’s Regulation on AI: What’s New and What’s Not?

About Morrison Forester

people-default-header-desktop.jpg

people-default-header-mobile.jpg

Default Meta Data

<p>Republished in <em>The Journal of Robotics, Artificial Intelligence & Law</em>.</p><p>The draft EU Regulation on Artificial Intelligence (the “<strong>Regulation</strong>,” available <a target="_blank" href="https://digital-strategy.ec.europa.eu/en/library/proposal-regulation-european-approach-artificial-intelligence">here</a>) imposes a broad range of requirements on both the public and private sectors, which are summarized in this <a href="/resources/insights/210421-high-risk-ai-systems.html">alert</a>. Some of these requirements already apply (in a similar form) under the EU General Data Protection Regulation (<strong>GDPR</strong>). This begs the question: What is the impact of the Regulation on the privacy sector, and what requirements already apply?</p><h4>GDPR requirements on the use of AI</h4><p>The GDPR applies to the processing of personal data in the context of an EU establishment, or when offering goods or services to, or monitoring the behavior of, individuals in the EU. The GDPR applies regardless of the <em>means</em> by which personal data are processed, and therefore applies when an AI system is used to process personal data (e.g., when using an AI system to filter applications for a job vacancy).</p><div class="float-md-right offset-md-1 col-md-5" style="border-top-width: 1px;border-top-style: solid;border-top-color: rgb(33, 37, 41);border-bottom-width: 1px;border-bottom-style: solid;border-bottom-color: rgb(33, 37, 41);"><blockquote class="mb-md-0 mt-5 mt-md-0 pt-3" style="font-size: 0.875rem;"><p>Under the GDPR, <em><strong>profiling</strong></em> is the use of personal data to evaluate certain personal aspects relating to a natural person, in particular to analyze or predict aspects concerning that natural person’s performance at work, economic situation, health, personal preferences, interests, reliability, behavior, location, or movements.</p><p><em><strong>Automated decision-making</strong></em> is any decision made without meaningful human involvement that has legal effects on a person, or similarly significantly affects him or her. This may partially overlap with, or result from, profiling, but this is not always the case.</p></blockquote></div><p>The GDPR imposes specific requirements on profiling and automated decision-making. The use of an AI system in relation to individuals often involves profiling, and sometimes automated decision-making. For example, when using an AI system to filter applications for a job vacancy, profiling is used to determine whether the applicant is a good fit for the vacancy. If the AI system filters out the applicants that it considers a good fit, and the other applicants are not considered for the position, this is automated decision-making towards the latter group. Their applications were removed from consideration for the position, with no meaningful human involvement.</p><h5>Legal requirements for users</h5><p>The GDPR imposes legal requirements on whoever uses the AI system for profiling and/or automated decision-making purposes, even if they acquired the system from a third party. These requirements include:</p><ul><li><em>Fairness</em>, which includes preventing individuals from being discriminated against; </li><li><em>Transparency</em> towards individuals, including meaningful information about the logic involved in the AI system; and</li><li><em>The right to human intervention</em>, enabling the individual to challenge the automated decision. </li></ul><h5>Contractual requirements for providers</h5><p>If a company acquires an AI system from a vendor, the company is often not in a position to comply with the above-mentioned requirements on its own. For example, the company may not know whether the AI is trained to prevent discrimination, or know the logic that the AI system relies on. In order to be able to comply with its obligations under the GDPR, a company therefore needs to rely on the vendor and will want to impose contractual obligations on the vendor to secure its cooperation and support.</p><h4>What’s new under the Regulation?</h4><p>Compared to the GDPR, the Regulation introduces new obligations for vendors of AI systems, prohibits certain very high-risk AI systems, and introduces more specific requirements for high-risk AI systems and users thereof. We highlight the key differences below.</p><h4>Extraterritorial scope</h4><div class="float-md-right offset-md-1 col-md-5" style="border-top-width: 1px;border-top-style: solid;border-top-color: rgb(33, 37, 41);border-bottom-width: 1px;border-bottom-style: solid;border-bottom-color: rgb(33, 37, 41);"><blockquote class="mb-md-0 mt-5 mt-md-0 pt-3" style="font-size: 0.875rem;">The Regulation defines <em><strong>AI systems</strong></em> as any software that, for a set of human-defined objectives, can generate outputs such as content, predictions, recommendations, or decisions influencing the environments where they interact. Such software qualifies as an AI system if it is developed using one or more of the following approaches and techniques:      
<ul><li><em>Machine learning approaches</em>, including supervised, unsupervised, and reinforcement learning, using a wide variety of methods including deep learning;</li><li><em>Logic- and knowledge-based approaches</em>, including knowledge representation, inductive (logic) programming, knowledge bases, inference and deductive engines, (symbolic) reasoning and expert systems; and/or</li><li><em>Statistical approaches, Bayesian estimation, and search and optimization methods</em>.</li></ul></blockquote></div><p>The Regulation applies to vendors (“providers”) of AI systems and the users of AI systems. A provider is the developer that offers an AI system on the market, whereas a “user” is using an AI system under its own authority. In respect of providers and users, the Regulation applies to:</p><ul><li><em>EU and non-EU providers</em> that place AI systems on the EU market;</li><li><em>EU users</em> of AI systems; and</li><li>Providers and users of <em>non-EU AI systems</em>, if the output of the AI system is used in the EU. </li></ul><h4>Prohibition on specific AI systems</h4><p>Although the GDPR imposes stringent requirements on certain processing activities, it does not outright prohibit any activities. This is different under the Regulation, which prohibits a number of AI systems that are deemed too risky under any circumstances. Most of these prohibitions are limited to AI systems used by public authorities or law enforcement. The prohibited AI systems that are relevant to the private sector are those that cause physical or psychological harm to an individual by:</p><ul><li>Deploying subliminal techniques to distort behavior; or</li><li>Exploiting vulnerabilities of a specific group of individuals due to their age or physical or mental disability.</li></ul><h4>High-risk AI systems</h4><p>The majority of the requirements of the Regulation apply to high-risk AI systems only. The Regulation lists a number of AI systems that qualify as high-risk. The European Commission can add AI systems to this list, taking into account the criteria set out in the Regulation. The key high-risk AI systems for the private sector are AI systems used for:</p><ul><li>“Real-time” and “after the fact” <em>remote biometric identification</em> of individuals (e.g., facial recognition); </li><li><em>Recruitment and selection</em>, such as advertising job vacancies and screening or filtering applications, and evaluating candidates in the course of interviews or tests;</li><li><em>HR purposes</em>, such as making decisions about promotions and terminations of work-related contractual relationships, for task allocation, and for monitoring and assessing performance and behavior; and</li><li>Evaluating <em>creditworthiness</em> of individuals or establishing a <em>credit score</em>.</li></ul><h4>General requirements for high-risk AI systems</h4><div class="float-md-right offset-md-1 col-md-5" style="border-top-width: 1px;border-top-style: solid;border-top-color: rgb(33, 37, 41);"><blockquote class="mb-md-0 mt-5 mt-md-0 pt-3" style="font-size: 0.875rem;">The Regulation imposes the following general requirements on high-risk AI systems:      
<ul><li><em>Establish a risk management system</em> and maintain it continuously throughout the lifetime of the system to identify and analyze known and foreseeable risks, estimate and evaluate such risks, and adopt suitable risk management measures;</li><li><em>Provide training, validation, and testing data</em>, including the relevance, representativeness, accuracy, and completeness thereof, and bias monitoring, detection, and correction, for which special categories of personal data may be used based on the substantial public interest exemption of Article 9(2)(g) GDPR;</li><li><em>Draw up technical documentation</em> before the AI system is placed on the market that demonstrates that the AI system complies with the Regulation;</li><li><em>Create automatic logs</em> to ensure a level of traceability of the system’s functioning;</li><li><em>Ensure transparency</em> to enable the user of the system to interpret the AI system’s output and use it appropriately;</li><li><em>Enable human oversight</em> on the AI system aimed at minimizing the risks to health, safety, or fundamental rights, by an individual who fully understands the system’s capabilities and limitations and can decide not to use the system or its output in any particular situation; and</li><li><em>Ensure accuracy, robustness, and cybersecurity</em> to foster resilience regarding errors, faults, inconsistencies, technical faults, unauthorized use, or exploitation of vulnerabilities.</li></ul></blockquote></div><p>The Regulation introduces general requirements for high-risk AI systems, which are more specific than the requirements under GDPR. For example, the Regulation imposes specific requirements on training, validation, and testing data in order to prevent bias and discrimination, while the GDPR merely requires that any processing of personal data is fair (including not being discriminatory). Another example is the requirement of human oversight. Although the GDPR grants individuals the right to obtain human intervention in cases of automated decision-making (as set out above), this requirement applies only to the company that makes the automated decision, and not to company that provided the (AI) system. This is different under the Regulation.</p><h4>Specific requirements for <em>providers</em> of high-risk AI systems</h4><p>The GDPR applies to the processing of personal data, and not (directly) to the provider of the systems that enabled this processing. This means that companies that use third-party systems to process personal data are responsible for such systems. This is different under the Regulation, which imposes the following specific requirements on the <em>provider</em> of the high-risk AI system:</p><ul><li><em>Ensure compliance</em> with the above-mentioned requirements for high-risk AI systems;</li><li><em>Implement a quality management system</em>, including a strategy for regulatory compliance, and procedures for design, testing, validation, data management, and recordkeeping;</li><li><em>Address (suspected) non-conformity</em> by immediately taking the necessary corrective actions to (i) bring the system into conformity, (ii) withdraw the system, or (iii) recall the system;</li><li><em>Notify relevant authorities about nonconformity of, or serious incidents pertaining to, the AI system</em> and the corrective measures taken in the countries in which the AI system is made available;</li><li><em>Conduct conformity assessments</em>, which can be internal or external assessments, depending on the type of high-risk AI system in use;</li><li><em>Register in the AI database</em> before offering a high-risk AI system on the market; and</li><li><em>Conduct post-market surveillance</em>, by collecting and analyzing data about the performance of high-risk AI systems throughout the system’s lifetime.</li></ul><h4>Specific requirements for <em>users</em> of high-risk AI systems</h4><p>The Regulation imposes fewer obligations on users of high-risk AI systems than on the providers thereof, which are different from the requirements that apply under the GDPR. The Regulation requires users of high-risk AI systems to:</p><ul><li><em>Abide by the provider’s instructions</em> on the use of the system, and take all technical and organizational measures indicated by the provider to address residual risks of using the high-risk AI system;</li><li><em>Ensure input data are relevant</em> if the user exercises control over such data, for example, that information on the applicant’s religion is not input into an hiring AI system;</li><li><em>Monitor the operation of the system</em> for anomalies or irregularities;</li><li><em>Maintain log files</em> if the logs are under the control of the user; and</li><li><em>Notify the provider about serious incidents and malfunctioning</em>, and, in such a case, suspend use of the AI system.</li></ul><h4>Specific requirements for certain AI systems</h4><p>The Regulation imposes specific requirements on AI systems for remote identification (such as facial recognition) in public spaces, which are only allowed with prior authorization of the competent authority. In addition, the Regulation imposes transparency requirements on AI systems that interact with individuals, recognize emotion, and create or alter image, audio, or video (e.g., “deepfakes”).</p><h4>European Artificial Intelligence Board and national authorities</h4><p>The Regulation provides for the establishment of a European Artificial Intelligence Board (the “<strong>Board</strong>”) tasked to issue guidance and opinions to ensure a consistent application of the Regulation, and to collect and share best practices and standards. In addition, each EU member state will designate a competent authority that is responsible for the implementation of the Regulation.</p><h4>Penalties</h4><p>The Regulation provides the following penalties for noncompliance in the private sector:</p><ul><li>Up to <em>EUR 30,000,000 or 6%</em> of the total worldwide annual turnover (whichever is higher) for noncompliance with the prohibited AI systems or the data and data governance requirements;</li><li>Up to <em>EUR 20,000,000 or 4%</em> of the total worldwide annual turnover (whichever is higher) for noncompliance with other than the above-mentioned obligations; and</li><li>Up to <em>EUR 10,000,000 or 2% </em>of the total worldwide annual turnover (whichever is higher) for providing incorrect, incomplete, or misleading information to competent authorities or conformity assessment entities.</li></ul><h4>Implementation, transition period, and next steps</h4><p>The Regulation provides for an implementation period of 24 months after entering into force, and a 12-month transition period for AI systems that are placed on the EU market before the application of the Regulation.</p><p>The Regulation is now under consideration by the European Parliament and the Council of the European Union, who will debate the proposal and can propose amendments. Together with the European Commission, these three legislative bodies will then work towards finalizing the Regulation, which is a time-consuming process.</p>