The European Commission today presented its Legislation on Artificial Intelligence. It is the first legal framework on this technology, which also comes accompanied by other regulations on machinery and robots. A new regulation on AI that wants to “guarantee security” and “strengthen investment in AI in the European Union”, creating various levels of risk and prohibiting facial recognition in certain situations.
Margrethe Vestager, European Commissioner for Competition, explains that this new legislation on AI will act “when fundamental rights are at stake”, pointing out that certain uses of AI are regulated but promising to leave room for innovation.
Here we explain what is the approach of the new framework on Artificial Intelligence, what is established for the use of technologies such as facial recognition and what questions are raised about this new regulation.
Europe sets risk levels for AI
The European Commission proposal to regulate artificial intelligence establishes four risk levels and informs that the norms must be implemented by all Member States equally, excluding the uses of AI at the military level from the regulations.
At the highest level is “unacceptable risk”. These are those AI systems considered a “threat to security, livelihoods and people’s rights”. Those AI systems that are included here will be prohibited.
Examples that the Commission sets out include, for example, an AI system that manipulates human behavior and incites violence, for example a voice-assisted toy that encourage dangerous behavior by minors. Also included here is a system of “social scoring” by governments to differentiate citizens.
1⃣ A legal framework on AI
We propose rules to make sure that #AI systems used in the EU are safe.
They will be categorised by risk:
⚫ Unacceptable
🔴 High risk
🟠 Limited risk
🟢 Minimal riskMore on risk categories → https://t.co/SD1ltmr9Vv#DigitalEU pic.twitter.com/u1S7rhnVJq
— European Commission 🇪🇺 (@EU_Commission) April 21, 2021
In a second point of “high risk” It includes uses of AI in critical infrastructures that can affect the health of citizens, AI to affect education and for example allows cheating in exams, components in surgery, personnel recruitment systems, public services, legislation, immigration o IA for administration or justice.
The European Union establishes some red lines and will ban certain uses of AI, such as “social scoring” systems by governments.
In all these areas, AI will need to be subject to strict obligations, including risk analysis, traceability of results, detailed documentation, human oversight and a high level of robustness.
At a lower level, “limited risk” include systems such as chatbots, which should have a minimum level of transparency and where users should be warned that they are talking to a machine.
The minimum risk includes the rest of the uses, for video games, image applications or other AI systems that do not involve risks. In these cases, the new regulations do not specify any measure.
Europe wants to promote the development of standards for AI and proposes to the different national organizations that they supervise this regulation. Additionally, the Commission invites the creation of voluntary codes of conduct for risk-free AI systems.
The European Parliament and the different governments of the European Union have yet to approve this new regulation, a process that could extend the implementation of these measures by more than a year.
How facial recognition is regulated
All “remote biometric identification” systems will be considered high risk. The European Commission has not decided to directly ban facial recognition systems, although it does describe that it will apply strict requirements such as those named above.
Facial recognition has its own category. Its use in public areas will be prohibited, but some exceptions are contemplated.
The Commission establishes that facial recognition will in principle forbidden to use in public areas and live as law enforcement. That is, the authorities will not be able to use facial recognition to prevent possible crimes, but there are some exceptions: “for example, when it is strictly necessary to search for a missing child, to prevent a specific and imminent terrorist threat or to detect, locate, identify or prosecute a perpetrator or suspect of a serious crime“.
These specific uses will be “subject to the authorization from a judicial body or other independent body and to the appropriate limits in the time, the geographic scope and the databases searched “.
The organization European Digital Rights (EDRi) expresses that it is “a great victory that the Commission recognize the need for red lines“.
(3) The ban on “remote biometric identification” applies only to law enforcement, has big exceptions & is narrow. Whilst happy to see some prohibition (national law required, case-case authorisations) this proposal does not go far enough to ban biometric mass surveillance.
– EDRi (@edri) April 21, 2021
Regarding facial recognition, EDRi believes that it was increasingly difficult to justify not prohibiting mass biometric surveillance and the fact of banning it on the street means that some requests have been heard. However, the ban has some exceptions and does not go “far enough.”
“Many of the most harmful uses are not prohibited, such as predictive policing, the uses of AI for migration control, the biometric categorization of race, gender, sexuality and also the surveillance of workers, which continue to be “high risk”, they explain from EDRi.
The Commission has chosen to create a specific section for facial recognition. A limbo between the category of “high risk” and that of “prohibited with exceptions”.
Among the doubts that arise is the definition of “remote” or “real time”, which could open the door to certain recognition systems such as via drones near the protesters, cameras in the police or systems that are not live. In this direction, we will have to wait until the implementation of the law in national legislation to better understand what kind of exceptions to facial recognition will be legal.
The Commission is encouraged to also regulate machinery such as robots or 3D printers
Together with the proposed regulation of AI, the Commission has announced the new regulation for machinery, which replaces the old directive of 2006. The new regulation extends the products included to cover products such as robots, lawnmowers , 3D printers, construction machines and industrial production lines.
The new Machinery Regulation establishes some additional requirements at the security level, “ensuring the safe integration of AI into machinery at a general level”, as explained by the Commission. This regulation will also try simplify the documentation required to obtain permits to be able to sell in the European Union.
More information | European Comission