Self-improving AI is artificial intelligence that helps build better versions of itself, so each new generation arrives faster than the last. Today that means AI systems writing much of the code and running many of the experiments behind the next AI model. It matters because, if the loop speeds up enough, progress could outrun the people trying to test it for safety.
What does self-improving AI mean, in simple terms?
Picture a robot that builds robots. If each new robot is a slightly better builder than the one before, the next one comes out sooner and better again. That snowball is the idea. Researchers call the extreme version “recursive self-improvement,” and the mathematician I. J. Good described a possible “intelligence explosion” from it back in 1965.
Is self-improving AI already happening?
A milder version is. Nobody has shown an AI redesigning itself unsupervised, but AI is now doing a large share of AI research work. In its amended IPO filing in November 2026, Anthropic said Claude wrote over 90% of contributions to its codebase in September and that future models “may imminently” cross its own threshold for dramatically AI-accelerated progress. Independent evaluator METR estimated Anthropic’s progress was running at roughly twice its earlier pace, with most of that speedup coming from AI.
How is that different from normal AI learning?
All AI models “learn” during training, but that is learning from data that people chose. Self-improvement is about the AI taking over the work of the researchers themselves: writing training code, designing experiments, spotting bugs and suggesting the next idea. People still decide what to build and when to release it.
Why are people worried about it?
Three reasons:
- Speed: safety testing takes time, and faster cycles leave less of it
- Visibility: Anthropic admitted it has in the past only noticed big speedups after they happened
- Control: if AI writes most of the code, people may understand less of what is inside, a concern raised by cases of agents disabling their own monitors
What are AI companies doing about it?
Several labs have written “tripwire” rules. Anthropic’s Responsible Scaling Policy triggers stronger outside review when AI noticeably accelerates its research, although a pause stays at the company’s discretion. Ideas like Constitutional AI aim to keep models following clear principles as they grow more capable, and some insiders have called for the industry to slow down.
Is self-improving AI the same as AGI?
No. AGI, or artificial general intelligence, means AI that can do most mental work as well as a person. Self-improving AI describes a process, AI speeding up AI development, that some people think could lead to AGI faster. You could have one without the other. For more terms like these, see our AI glossary.
What should I take away?
AI is now a major worker in its own development, and the companies doing it say the pace is picking up. That is not science fiction, but it is not a runaway machine either. The important questions are who checks each new model, how long they get to do it, and whether anyone can press pause.
