AI: MEPs climb down on request for unconditional ban on live facial recognition

The EU bloc reached a provisional agreement on an Artificial Intelligence Act which legislators said would ensure AI in Europe is safe and respects fundamental rights and democracy.

But the law has been lambasted by Amnesty International for not having banned public mass surveillance.

On Friday, European Parliament and Council negotiators reached a provisional agreement on the AI Act, to ban applications that carry biometric categorisation that use sensitive characteristics, such as political, religious, philosophical beliefs, sexual orientation, race; untargeted scraping of facial images from CCTV footage; emotion recognition in the workplace and schools; social scoring; and systems that manipulate human behaviour or exploit people’s vulnerabilities.

But it was also apparent that the European Parliament climbed down from its original position on strong protections, including an unconditional ban on live facial recognition.

Proponents argue that the draft allows only limited use of facial recognition and is subject to safeguards. But Amnesty insisted that no safeguards prevent the human rights harms that facial recognition inflicts.

“Not ensuring a full ban on facial recognition is therefore a hugely missed opportunity to stop and prevent colossal damage to human rights, civic space and rule of law that are already under threat throughout the EU,” Amnesty AI advisor Mher Hakobyan, said.

“Lawmakers also failed to ban the export of harmful AI technologies, including for social scoring, which would be illegal in the EU. Allowing European companies to profit abroad off from technologies that the law recognises impermissibly harm human rights in their home states establishes a dangerous double standard,” Amnesty added.

Law enforcement exemptions

MEP negotiators agreed on a series of safeguards and narrow exceptions for the use of biometric identification systems (RBI) in publicly accessible spaces for law enforcement purposes, subject to prior judicial authorisation and for strictly defined lists of crime.

“Post-remote” RBI would be used strictly in the targeted search of a person convicted or suspected of having committed a serious crime. “Real-time” RBI would comply with strict conditions and its use would be limited in time and location, for the purposes of: targeted searches for victims of trafficking or abduction, prevention of a terrorist threat, and localising criminal and terror suspects – specifically in crimes of trafficking, sexual exploitation, murder, kidnap, rape, armed robbery, organised crime and environmental crimes.

Obligations for high-risk systems

MEPs successfully managed to include a mandatory fundamental rights impact assessment, among other requirements, applicable also to the insurance and banking sectors.

For example, AI systems used to influence the outcome of elections and voter behaviour, are classified as high-risk and therefore subject to clear obligations.

Citizens will have a right to launch complaints about AI systems and receive explanations about decisions based on high-risk AI systems that impact their rights.

Non-compliance with the rules can lead to fines ranging from €35 million or 7% of global turnover to €7.5 million or 1.5% of turnover, depending on the infringement and size of the company.

 “Thanks to the European Parliament’s resilience, the world’s first horizontal legislation on artificial intelligence will keep the European promise – ensuring that rights and freedoms are at the centre of the development of this ground-breaking technology,” said Italian co-rapporteur Brando Benifei (S&D).

Co-rapporteur Dragos Tudorache (Renew, Romania) said: “The EU is the first in the world to set in place robust regulation on AI, guiding its development and evolution in a human-centric direction. The AI Act sets rules for large, powerful AI models, ensuring they do not present systemic risks to the Union and offers strong safeguards for our citizens and our democracies against any abuses of technology by public authorities.”

Intensive technical meetings are set to follow the provisional high-level political deal before a finalised text can be adopted as law.