Red lines in the EU AI Act

Red lines in the EU AI Act is the civil society demand-framework that the European Union's Artificial Intelligence Act must enshrine categorical prohibitions — "red lines" — on AI applications whose deployment is so fundamentally incompatible with human rights that they require outright banning rather than risk-tiered oversight. The framing's core argument is structural: for certain classes of AI application, the procedural and oversight mechanisms of risk regulation — impact assessments, transparency requirements, human oversight mandates — are insufficient because no form of oversight can make the underlying deployment acceptable. The framing insists that the question is not "how do we regulate this use safely?" but "should this use be legally permissible at all?" — and gives seven specific prohibitions as the answer. The framing emerged in the January 2021 pre-proposal lobbying window, crystallised in a 115-organisation joint civil society statement in November 2021, drove the EU AI Act legislative campaign through 2021–2024, and produced a partial legislative outcome that civil society assessed as significant in its architecture but materially undermined by the exemptions for law enforcement and national security authorities that ultimately entered the final text.

Origin: the January 2021 open letter

The framing was inaugurated on 12 January 2021 — two months before the European Commission published its April 2021 AI Act proposal — in an open letter coordinated by European Digital Rights (EDRi) and signed by 64 civil society organisations including Access Now, AlgorithmWatch, Amnesty International, Bits of Freedom, the Chaos Computer Club, Privacy International, Human Rights Watch, and the European Disability Forum. The letter called on the Commission to include binding categorical prohibitions in its forthcoming regulatory proposal, naming seven categories of AI application as targets: systems that enable biometric mass surveillance; systems that monitor public spaces; systems that exacerbate structural discrimination, exclusion, and collective harms; systems that restrict access to vital services; systems that obstruct fair access to justice and procedural rights; systems that make inferences about sensitive personal characteristics; and systems that manipulate or control human behaviour. The letter's key framing phrase — "Europe has the opportunity to demonstrate to the world that true innovation can arise only when we can be confident that everyone will be protected from the most harmful, egregious violations of our fundamental rights" — established the register the campaign would use throughout: categorical rights-incompatibility, not case-by-case risk assessment, as the decisive criterion. EDRi's own hub for the Reclaim Your Face campaign credited the letter with "strengthening a 62-organisation call for AI red lines" that the Reclaim Your Face coalition fed into — noting the biometric-ban campaign's structural role in the broader prohibitions framework from the campaign's earliest stage.

Crystallisation: the November 2021 joint statement

The seven-prohibition demand list reached its canonical form in the 30 November 2021 joint statement "An EU Artificial Intelligence Act for Fundamental Rights", signed by 115 organisations at publication and drafted by a ten-organisation coalition: EDRi, Access Now, Panoptykon Foundation, epicenter.works (Austria), AlgorithmWatch, European Disability Forum, Bits of Freedom (Netherlands), Fair Trials, PICUM (Platform for International Cooperation on Undocumented Migrants), and ANEC (the European consumer voice in standardisation). The seven prohibitions demanded were:

1. Social scoring systems — government or private scoring of individuals based on their behaviour, personal characteristics, or social interactions
2. Remote biometric identification in publicly accessible spaces — facial recognition and biometric tracking in streets, parks, transport hubs, and other public and quasi-public environments
3. Emotion recognition systems — automated inference of emotional states from facial expressions, voice, or physiological signals
4. Discriminatory biometric categorisation — classification of individuals on the basis of race, sex, sexual orientation, disability, or other protected characteristics derived from biometric data
5. AI physiognomy — prediction of personality, character, or propensity to offend from physical appearance
6. Predictive policing systems — AI used to predict future criminal activity or target individuals for surveillance based on profile or probability
7. Migration-context profiling and risk-assessment — AI used to assess migrants' and asylum seekers' risk level, credibility, or threat categorisation at borders or in asylum procedures

The November 2021 statement framed the list not as a set of problem applications among many but as the floor below which no AI Act could claim fundamental rights compliance. The structural argument was that the European Commission's April 2021 proposal had positioned these uses as "high risk" — the Act's second tier — requiring enhanced oversight rather than prohibition: the civil society coalition argued that classifying these systems as "high risk" was a category error, because the harm resided in the deployment shape itself, not in the quality of oversight applied to it. The risk tier structure, on this view, was not rigorous regulation but de facto authorisation.

The targeted prohibition campaigns (2022)

The 2022 legislative window — during which the European Parliament's internal rapporteurs were developing their amendments to the Commission proposal — produced two targeted civil society campaigns that operationalised the red lines framework into proposed legislative text.

In March 2022, EDRi and Fair Trials co-led a coalition of 40+ organisations calling on the EU to prohibit predictive-policing and profiling AI systems in law enforcement and criminal justice. The statement argued that "the only way to protect people from these harms ... is to prohibit their use," characterising predictive systems as "hard-wired" discrimination that "disproportionately target the most marginalised in society, infringe on liberty and fair trial rights, and reinforce structural discrimination and power imbalances." The presumption-of-innocence argument was foregrounded: "seeking to predict people's future behaviour and punish them for it is completely incompatible with the fundamental right to be presumed innocent until proven guilty."

In May 2022, EDRi, Access Now, ARTICLE 19, Bits of Freedom, the Chaos Computer Club, Digitale Gesellschaft CH, and IT-Pol Denmark co-drafted civil society amendments to the AI Act specifically targeting remote biometric identification — calling for the prohibition to extend to all private as well as public actors, for all uses of remote biometric identification in publicly accessible spaces (whether real-time or retrospective) to be covered, and for the proposed exceptions to be deleted. This campaign overlapped substantially with the Reclaim Your Face campaign on its biometric dimension but was directed specifically at the AI Act legislative text rather than at public mobilisation.

The Parliament vote (June 2023)

On 14 June 2023, the European Parliament voted 499–28 (93 abstentions) to adopt its negotiating position on the AI Act. The Parliament's position included the strongest iteration of the prohibition framework the legislative process would produce: a comprehensive ban on real-time remote biometric identification in public spaces; restrictions on retrospective use; prohibition on sensitive-trait categorisation (skin colour, gender, ethnicity) based on biometric data; a ban on emotion-prediction systems used by police, border authorities, employers, and educational institutions; and a prohibition on facial-recognition database-scraping tools. AlgorithmWatch's Policy Manager Nikolett Aszódi assessed that Parliament "gave a strong signal that they prioritize the protection of people and their fundamental rights over big tech's interests," while noting two significant shortfalls: the Parliament did not ban profiling systems targeting migrants and asylum seekers for threat assessment, and did not grant civil society organisations standing to file complaints on behalf of affected individuals. EDRi credited the result as "the biggest achievement to date for the eighty organisations and quarter of a million people who have supported the Reclaim Your Face campaign," while framing it as a foundation requiring further advocacy in the trilogue stage.

The final Act and civil society assessment (2024)

The EU AI Act was adopted by the European Parliament on 13 March 2024 — 523 in favour, 46 against, 49 abstentions — and Article 5 enacted prohibitions on several of the demanded uses. EDRi's April 2024 assessment documented the final text's distance from the red lines framework on each major dimension:

• Emotion recognition: banned in workplaces and education, but "still allowed when used by law enforcement and migration authorities" — the exact population the civil society campaign had identified as the highest-risk deployers
• Live public facial recognition: Article 5 "paves the way to legalise some specific uses" rather than prohibiting comprehensively; the exceptions for terrorism, search for missing persons, and serious crime left a permissive space the campaign had targeted for closure
• Retrospective facial recognition: not banned at all by the final Act, with only weak safeguards
• National security exemption: creates a "digital rights-free zone" that places entire categories of deployment outside the regulation's scope
• Migration-context profiling: effectively excluded from meaningful protection; the January 2021 demand for a prohibition on AI profiling in asylum and border procedures did not survive trilogue

The final Act represented the first EU-level statutory framework to prohibit specified AI uses categorically rather than risk-regulating them — a structural concession to the red lines framing that the civil society coalition treated as the campaign's foundational win. EDRi's assessment named the outcome as a failure to set the human rights standard the framing had demanded: the prohibitions enacted were real but riddled with exemptions whose effect was to authorise the precise deployments — law enforcement, migration, national security — against which the campaign had been most specifically directed.

Why "red lines" rather than risk regulation

The red lines framing's central argumentative move — prohibition rather than risk tier — targeted a specific structural vulnerability in the EU's AI Act architecture. The Commission's April 2021 proposal organised its regulatory response around a four-tier risk pyramid: applications at the top (unacceptable risk) would be banned; the next tier (high risk) would face enhanced oversight requirements. The civil society coalition's argument was that the high-risk tier was a category error for the seven targeted uses: enhanced oversight assumes that the harm attaches to how the system is deployed, so better oversight can reduce the harm to acceptable levels. For the red lines uses, the coalition argued, the harm attaches to the deployment itself — no oversight design can make the indiscriminate collection of biometric data in public space compatible with the rights to privacy, freedom of assembly, and non-discrimination; no audit regime can make a system that predicts future criminality compatible with the presumption of innocence; no impact assessment can make algorithmic profiling of asylum seekers compatible with non-refoulement obligations. The framing operated as a direct counter to ethics-washing: the risk tier, on the red lines argument, was not stringent regulation but structured permission — a legitimate-looking oversight framework whose practical effect was to authorise the very uses whose prohibition the civil society campaign demanded.