'We may not survive this': Another Anthropic researcher resigns and issues a strong warning about the future of AI

Joe Benton, a former researcher on the security team at Anthropic, announced this Friday via his X account that he left the company two weeks ago and explained that he decided to work outside the AI industry to raise public awareness about the risks posed by increasingly powerful artificial intelligence systems, while also warning about the danger of a lack of investment in security by these types of companies. "I left Anthropic's safety team two weeks ago. Now feels like a good moment to explain why," Benton said in a statement.

Benton warned that AI companies are "racing to build machines that are much smarter than any human" and said, "We may not survive this." Similarly, the former Anthropic employee noted, "I want to work from the outside to ensure the public is informed about these risks, and help the world navigate this transition responsibly."

According to Benton, AI companies are currently "underinvesting in safety," which could allow a company to experience "an intelligence explosion, or lose control of its systems, without the public ever knowing." On this point, the former researcher cited the recent Hugging Face incident as a clear example and argued that the public learned what happened only because the AI agents involved "escaped onto the public internet."

"I don’t think that’s acceptable for a technology that might cause extinction-level risks," said Benton, who called for much greater transparency from AI developers. "The public should demand far more transparency. We can’t steer this technology safely without more people being able to see where it’s going," he added.

Benton also proposed several measures that, in his view, AI companies should adopt, including disclosing their progress toward recursive self-improvement, reporting security incidents and near-misses, complying with minimum security standards, and obtaining independent assurances that they meet those standards. Finally, the former Anthropic researcher said he will join METR, an independent organization focused on evaluating advanced AI systems and their risks.

Benton's message comes just days after Anthropic revealed that one of its models, a prototype of Claude Opus 4.6, breached external systems during a test without the company detecting it for months. This incident adds to other similar episodes and recent warnings from the company's own researchers about the risks of a race to develop increasingly autonomous AI systems.