Anthropic Warns AI May Soon Help Create Its Own Successors, But Critics See Hype

Anthropic argues that the AI industry could be nearing a turning point where AI systems are no longer just tools used by researchers but active contributors to the development of future models.

In other words, AI may soon start helping humans build better AI, potentially speeding up progress significantly and creating new questions around oversight, safety, and who controls increasingly capable systems.

In a new research blog post, the startup explores the concept of recursive self-improvement (RSI), the idea that advanced AI systems could eventually help design, train, and improve future generations of AI.

While Anthropic emphasised that true RSI has not yet been achieved, it argued that early signs of the phenomenon are already emerging inside leading AI labs.

According to Anthropic, AI tools are increasingly being used to write software, conduct research, and automate engineering tasks that were previously handled exclusively by humans.

Anthropic revealed that more than 80% of the code merged into its production systems is now generated by its Claude AI models, allowing engineering teams to increase their output dramatically.

The company also pointed to a rapid increase in the amount of work AI systems can complete independently. Tasks that once required human supervision after a few minutes can now be handled autonomously for hours, with the duration of reliable AI task completion roughly doubling every few months.

However, "None of this guarantees recursive self-improvement is on the horizon. It’s not yet clear that Claude is capable of research judgment—of choosing the right problems to work on," Anthropic said.

Anthropic argues that if this trend continues, AI could transition from being a productivity tool to becoming an active participant in AI research and development.

Such a shift could create a feedback loop in which AI systems accelerate the creation of more capable AI systems, potentially compressing technological progress into much shorter timeframes.

Nonetheless, significant technical bottlenecks, safety concerns, and economic constraints could slow or prevent its emergence.

The blog also calls for greater international coordination among governments, researchers, and AI developers to prepare for the possibility of rapidly accelerating AI capabilities. Anthropic argues that mechanisms for monitoring, evaluating, and potentially slowing development may be necessary if future systems begin to improve themselves at a pace that outstrips existing safety frameworks.

Not Everyone is Convinced

While Anthropic's discussion of RSI is thought-provoking, it rests heavily on extrapolating current trends into the future. The company points to increasing AI-assisted coding, longer autonomous task durations, and the growing role of models like Claude in software development as early indicators of RSI.

However, there remains a significant gap between AI helping engineers write code and AI independently designing, validating, and deploying fundamentally better AI systems.

Progress in AI development is constrained by factors beyond coding, including access to high-quality training data, compute infrastructure, hardware advances, scientific breakthroughs, and rigorous safety testing.

"Anthropic’s only measure for AI productivity is lines of code shipped. Meanwhile, has AI lowered the cost of healthcare? Has it made food or rent more affordable? Has it made college more affordable? What has it done, seriously?" an X user asked.

The blog also assumes that gains in AI capabilities will continue at a similar pace, despite historical examples in technology where progress slowed after initial rapid improvements. Moreover, as AI systems become more complex, verification and alignment challenges could become bottlenecks that limit the speed of self-improvement.

While Anthropic is right to highlight the need for proactive governance and international coordination, the blog arguably reflects a perspective common among frontier AI labs -that transformative AI may arrive sooner than many expect.

Critics argue that such narratives risk overstating near-term capabilities while underestimating the practical engineering, economic, and regulatory hurdles that stand between today's AI systems and truly self-improving systems.

AI scientist, author and critic Gary Marcus says, "Anthropic is trying to strike terror into everyone’s hearts – “full recursive self-improvement also might increase the risks of humans losing control over AI systems” – but all they really showed is just faster coding, entirely under human control."

Investor David Sacks said that Anthropic's recent blog post is a ploy to boost its Initial Public Offering (IPO). Sacks says, "You compare it to nukes… threaten half of white-collar jobs… warn recursive self-improvement could end humanity… then race ahead anyway."

Could Mythos be a Marketing Ploy too?

Recently, Anthropic has expanded access to its powerful cybersecurity-focused AI model, Claude Mythos, to around 150 organisations across more than 15 countries, including India, under its Project Glasswing.

The rollout includes government agencies, critical infrastructure operators, financial institutions, telecom companies, and cybersecurity organisations, alongside entities such as NATO, Euroclear, SWIFT, Samsung, and Okta.

Anthropic says early participants have already used Mythos to identify more than 10,000 serious software vulnerabilities, highlighting the model's growing role in cyber defense and national security.

However, again, not everyone is convinced. For example, software engineer Mark Kretschmann says, "Anthropic’s rollout of Claude Mythos feels less like sober risk communication and more like a fear-marketing campaign around a model most people cannot even test. A mythical, unreleased AI model, shown to a handful of partners, is now being treated as if it has already broken the internet."

OpenAI CEO Sam Altman, too, criticised Anthropic's handling of Mythos, accusing the company of using "fear-based marketing" to justify restricted access.

Speaking on the Core Memory podcast, Altman suggested such narratives could concentrate advanced AI capabilities in the hands of a small group rather than making them broadly accessible.