Anthropic has urged policymakers and major AI developers to develop a coordinated emergency mechanism that could pause frontier AI development if future systems begin advancing faster than society can safely manage. The proposal is not a call to halt AI research immediately. Instead, the company argues that governments and leading AI laboratories should agree in advance on a framework that would allow development to be temporarily slowed if advanced AI systems start improving themselves at a pace faster than human oversight, regulation, and safety measures can manage.
The warning is based on a trend Anthropic claims it is already seeing inside its own operations. According to the Dario Amodei-led firm, more than 80% of the code merged into its software systems during May was written by Claude, its flagship AI model. AI is increasingly handling programming, debugging, testing, and research tasks that were previously performed by human engineers. Anthropic states these tools are no longer just productivity assistants – they are becoming active contributors to the development process itself. As AI-generated code quality improves and models become better at research and problem-solving, the company believes AI could play a growing role in building future generations of AI systems.
“The Anthropic Institute will conduct research—in collaboration with many others—and take actions to help build the systems that a credible slowdown or pause would require,” the AI firm said.
At the center of the discussion is a concept known as recursive self-improvement. In this scenario, humans build an AI model that helps create a better model, and the improved model then helps create an even more capable successor. Each cycle could potentially happen faster than the one before it.
Anthropic is not claiming that fully autonomous self-improving AI exists currently, but it argues that the industry is moving closer to a world where AI systems contribute significantly to their own advancement. And if that happens, the pace of progress could increase dramatically, making it difficult for governments, regulators, safety researchers, and society to keep up.
Importantly, the company’s concern is not limited to technical risks. Anthropic argues that rapid AI progress could outpace society’s ability to adapt economically and politically. AI companies are investing hundreds of billions of dollars in computing infrastructure, specialized chips, and data centers. At the same time, AI tools are spreading across industries including software engineering, customer support, finance, legal services, education, healthcare, and scientific research.
Anthropic CEO Dario Amodei has repeatedly warned that advanced AI could automate large amounts of white-collar work and reshape labour markets much faster than previous technological revolutions. The company believes institutions that typically evolve over decades may struggle to respond to changes occurring over months. Considering these developments, Anthropic stressed that any meaningful pause in AI development would need to be coordinated internationally. The company argued that if one AI laboratory slowed down while competitors continued developing more powerful systems, little would be achieved in terms of reducing overall risks.
The Tech Portal is published by Blue Box Media Private Limited. Our investors have no influence over our reporting. Read our full Ownership and Funding Disclosure →
Ashutosh is a Senior Writer at The Tech Portal, largely reporting on new tech, and intersection of technology and business. Ashutosh’s career spans across nearly a decade of technology writing across multiple platforms and languages.
