"A Right to Warn about Advanced Artificial Intelligence" — current and former AI lab employee open letter (4 June 2024)

On 4 June 2024, current and former employees of OpenAI and Google DeepMind published "A Right to Warn about Advanced Artificial Intelligence" — an open letter demanding structural whistleblower protections inside frontier-AI companies. Seven researchers signed by name: Jacob Hilton, Daniel Kokotajlo, Ramana Kumar, Neel Nanda, William Saunders, Carroll Wainwright, and Daniel Ziegler. Six more signed anonymously — four current and two former OpenAI employees. The letter also carried the explicit endorsement of Turing Award winners Geoffrey Hinton and Yoshua Bengio and computer scientist Stuart Russell. It was the first time current employees of OpenAI had publicly and collectively raised concerns about the company's safety culture in a structured advocacy document, even if only anonymously, and it arrived ten days after Sam Altman had been forced by public pressure to retract the nondisparagement agreements that had been its immediate catalyst.

Background: the superalignment departures and the NDA controversy

The letter emerged from a cascade of events at OpenAI in May 2024 that made the company's internal safety culture unusually visible. On 15 May 2024, Ilya Sutskever — OpenAI's co-founder and chief scientist, and one of the two heads of its Superalignment team — announced he was leaving. Two days later, on 17 May 2024, Jan Leike, the other head of the Superalignment team, resigned and published a thread on X that stated: "Over the past years, safety culture and processes have taken a backseat to shiny products." Leike wrote that he had "reached a breaking point" and that the company should be spending far more bandwidth preparing for the next generation of models. OpenAI confirmed that it had dissolved the Superalignment team, integrating its members across other research groups.

In the weeks that followed, reporting in major outlets revealed that OpenAI had been requiring departing employees to sign unusually broad nondisparagement agreements — agreements whose language threatened to claw back vested equity if the former employee publicly "disparaged" the company, a provision understood to cover statements about safety concerns drawn from nonconfidential public information. The controversy was acute enough that on 24 May 2024, Sam Altman sent an internal memo stating that the nondisparagement clauses would be removed from standard departure paperwork, that former employees would be released from existing obligations, and acknowledging that the equity-clawback language had been a mistake. The letter followed ten days later.

The letter's demands

The letter named three AI risk categories it treated as warranting public concern: further entrenchment of existing inequalities; manipulation and misinformation; and loss of control of autonomous AI systems, with the potential for human extinction. It did not stake a position on the likelihood or imminence of any of these risks. Its argument was structural: researchers inside frontier-AI labs are among those best positioned to assess and surface such risks, and the current institutional architecture — confidentiality obligations, equity-contingent silence requirements, cultural norms against public criticism — provides no adequate channel for them to do so.

Four principles organized the letter's demands:

1. No nondisparagement enforcement on safety matters. Companies must not enforce agreements that prohibit criticism related to safety risks, and must not retaliate against employees for raising such concerns internally.
2. Anonymous reporting channels. Organizations should establish verifiably anonymous channels enabling employees to report safety concerns to boards, regulators, and independent expert bodies.
3. Culture of open criticism. Companies should foster cultures in which workers can publicly discuss risk-related concerns while protecting legitimate intellectual-property interests.
4. Whistleblower protection for public disclosure. Organizations must not punish employees who share risk-related confidential information publicly after internal reporting mechanisms have proven inadequate.

The structural framing — demanding channels and protections rather than specific policy changes — allowed the letter to be signed by named researchers across OpenAI and Google DeepMind simultaneously, and to attract the endorsement of Hinton, Bengio, and Russell, whose standing in the research community rests on institutional independence rather than company affiliation.

Response

OpenAI's spokesperson responded that the company was proud of its "track record providing the most capable and safest AI systems" and emphasised the importance of rigorous debate. The statement did not engage directly with the letter's structural demands. Altman's 24 May memo had addressed the most concrete grievance — the equity-clawback mechanism — but critics noted that the letter's four principles went well beyond NDA reform to cover the broader institutional culture around retaliation and anonymous reporting.

Significance

The Right to Warn letter is the corpus's clearest example of insider mobilization within the frontier-AI safety community — an action by people inside the labs themselves, using the form of a structured advocacy document co-signed by eminent external validators to apply institutional pressure on their employers. Its closest structural precedent in the corpus is the forced departure of Timnit Gebru from Google's Ethical AI team in December 2020 — also a moment when insider research-community concern about a major lab's treatment of safety-relevant work became a public advocacy moment. Both events share the same pattern: a departure precipitated by a lab's management of safety-adjacent research, a public airing of institutional-culture concerns by those closest to the work, and a downstream institutional response that addressed the immediate grievance while leaving structural questions open.

The two events differ on the terrain and the form. The Gebru event centred on algorithmic accountability and racial bias; its downstream institution was DAIR. The Right to Warn letter centred on existential-risk concerns and produced a structural-demands document — naming the four protections without attempting to found an institution. Together they trace the expansion of the inside-researcher advocacy form across the early 2020s: from algorithmic-accountability researchers departing Big Tech and building independent institutions, to a broader class of safety-focused researchers inside frontier-AI labs using the open letter as a public-pressure instrument while remaining inside or recently departed from those labs.

The backing of Hinton and Bengio — two of the three 2018 Turing Award winners, and the researchers most publicly associated with the case that advanced AI poses serious risks — gave the letter's structural framing a mainstream legitimacy the employee signatories alone could not have provided, and moved coverage from specialist tech press to major broadcast and print outlets. OpenAI's pre-emptive NDA reversal ten days before publication suggests that the inside-out pressure the letter represented had already begun to shift the company's institutional posture before the document was publicly released.