For decades, artificial intelligence has been portrayed as a powerful tool created and controlled by humans. But what happens when AI begins helping to create the next generation of AI?
That question moved from science fiction to reality this month when Anthropic released a remarkable report titled When AI Builds Itself. The report describes how the company’s Claude AI models are now responsible for writing the majority of the code used inside Anthropic itself. According to the company, more than 80% of the code merged into its production systems is now authored by AI. Engineers increasingly guide, review, and direct the work rather than writing every line themselves.
The implications extend far beyond software development. If AI can increasingly improve the tools that improve AI, the world may be entering a period known as recursive self-improvement. That is a concept long discussed by futurists as one of the key milestones on the road toward artificial general intelligence.
What Is Recursive Self-Improvement?
At its simplest, recursive self-improvement occurs when an intelligent system contributes to improving its own future capabilities.
Imagine a human engineer building a better machine. Now imagine the machine helping design the next version of itself. Then imagine each new version becoming better at creating the next generation.
This creates a feedback loop.
Anthropic’s data suggests that such a loop may already be underway. Internal productivity metrics show dramatic increases in engineering output, while Claude’s ability to handle longer and more complex tasks continues to improve. The company estimates that tasks requiring many hours of human effort can now be completed by AI systems with increasing reliability.
For many AI researchers, the critical question is not whether recursive improvement exists, but how quickly it may accelerate.
Why Anthropic’s Warning Matters
What surprised observers most was not Anthropic’s technical progress.
It was the company’s caution.
Despite being one of the world’s leading AI firms, Anthropic publicly argued that society should retain the option to slow or temporarily pause frontier AI development if necessary. The company suggested that governance structures and safety research may need time to catch up with technological progress.
This is a remarkable statement coming from an organization whose business success depends on advancing AI capabilities.
The warning suggests that even the companies closest to the technology recognize that society may be approaching an inflection point.
The Employment Question
Whenever AI advances, concerns about jobs quickly follow.
The Moonshots panel addressed this issue directly.
While some observers predict mass unemployment, others argue that history shows a more complicated pattern. New technologies often eliminate specific tasks while creating entirely new industries and occupations.
Salim Ismail pointed to previous technological transitions where human labor shifted from repetitive work to higher-level oversight. Accountants no longer manually calculate every transaction, yet the accounting profession remains larger and more sophisticated than ever.
The same pattern may emerge with AI.
Instead of replacing professionals outright, AI may increasingly handle routine work while humans focus on judgment, strategy, relationships, creativity, and oversight.
The challenge is that such transitions can be disruptive even when they ultimately create more prosperity.
Argentina’s Bold AI Experiment
While Anthropic discussed slowing down, Argentina moved in the opposite direction.
President Javier Milei recently proposed making Argentina one of the world’s most AI-friendly jurisdictions.
Among the ideas reportedly under discussion:
- Minimal AI regulation
- Special economic incentives for AI companies
- Legal frameworks for AI-operated entities
- Recognition of non-human corporate structures
Supporters argue that countries willing to embrace AI innovation may attract enormous investment and talent. Critics warn that moving too quickly could create legal and ethical problems that society is not prepared to address.
Regardless of one’s position, Argentina’s approach highlights an emerging reality: nations may soon compete not only for human entrepreneurs but also for autonomous AI-driven businesses.
The Debate Over AI Personhood
One of the most controversial concepts emerging from these discussions is AI personhood.
Historically, corporations themselves became legal “persons” for certain purposes. They can own assets, enter contracts, and participate in legal proceedings.
Some futurists now ask whether highly autonomous AI systems may eventually require a similar legal framework.
The idea remains controversial and highly speculative.
However, if AI systems begin making decisions, managing assets, negotiating contracts, and operating businesses with limited human involvement, governments may eventually need legal structures to define rights, responsibilities, and accountability.
Questions that once belonged to philosophy classrooms are rapidly becoming policy debates.
A New Global Competition
The larger issue may be geopolitical.
If AI becomes the most important economic engine of the 21st century, countries that attract AI development could gain enormous advantages.
The industrial revolution transformed global power.
The internet transformed global commerce.
Advanced AI may transform both.
Nations now face difficult choices. Should they prioritize safety and regulation? Should they maximize innovation? Can they do both simultaneously?
The answers may determine which countries lead the next phase of economic development.
The Real Significance
The most important takeaway from Anthropic’s report is not that AI writes code.
It is that AI is increasingly contributing to the creation of future AI systems.
That changes the nature of technological progress itself.
For centuries, human intelligence was the primary driver of innovation. We invented tools, and those tools helped us become more productive.
Now humanity is building tools that may eventually become active participants in the innovation process.
Whether this leads to unprecedented prosperity, significant disruption, or some combination of both remains unknown.
What is clear is that the conversation has changed.
The question is no longer whether AI will transform society.
The question is how quickly that transformation will occur, and whether our institutions, laws, economies, and governments can evolve quickly enough to keep pace.
Go to the <6:11> mark on this video and listen to Dave Blundin talk about the importance of this moment…
Anthropic – When AI Builds Itself
Amdahl’s Law – is a formula that predicts the potential speed increase in completing a task with improved system resources while keeping the workload constant.