Recursive Self-Improvement Is the New AGI: A Goal That's Proving Just as Slippery

Picture a neural network rewriting its own code, identifying its weaknesses, and patching them overnight. By morning, it's measurably smarter. By next week, it's surpassed its creators. This is the vision of recursive self-improvement (RSI), and it's attracting serious capital from a new generation of AI labs.

The problem is that we've heard this story before. Actually, the research shows we've heard variations of it for decades, and the gap between the vision and demonstrated results remains, to be precise, enormous.

Background

Recursive self-improvement refers to AI systems capable of autonomously enhancing their own capabilities. The concept isn't new. I. J. Good's 1965 paper on "intelligence explosions" outlined the basic logic: a sufficiently intelligent machine could design a better version of itself, which could design an even better version, and so on. The result, Good argued, would be runaway superintelligence.

What's new is the money. According to Bloomberg, a growing number of AI companies are now explicitly targeting RSI as their primary research objective. TechCrunch reports that RSI has become "the new AGI" in Silicon Valley pitch decks, a north star goal that promises transformative returns while remaining conveniently difficult to define or falsify.

The timing isn't coincidental. Large language models have demonstrated surprising emergent capabilities, and the scaling hypothesis (more compute equals more capability) has held up better than many researchers expected. If you squint, you can see a path from current systems to ones that might meaningfully contribute to their own training. The operative word being "squint."

What's actually being claimed

The companies pursuing RSI vary in their ambitions and their honesty about the challenges involved. Some are focused on narrow self-improvement: systems that can identify gaps in their training data and suggest what additional information would be useful. Others are pursuing something closer to Good's original vision, systems that can modify their own architectures and training procedures.

It's worth noting that even the narrow version faces significant obstacles. Current language models can certainly generate code, including code that modifies AI systems. But generating code and generating better code are different things. The evaluation problem is severe: how does a system know whether a proposed modification is actually an improvement? Human researchers struggle with this constantly. We run experiments, we wait for results, we argue about metrics. The idea that an AI system could shortcut this process assumes solutions to problems we haven't solved ourselves.

I know I'm being picky here, but the distinction matters. There's a meaningful difference between "AI that assists in AI research" (which already exists and is useful) and "AI that autonomously improves itself in a recursive loop" (which remains speculative). Much of the current RSI discourse conflates these two categories, intentionally or otherwise.

The research gap

Here's what concerns me about the current RSI push: the theoretical foundations are surprisingly thin. We have Good's original argument, which is more of a thought experiment than a research program. We have various formalisations in the AI safety literature, most of which focus on why RSI might be dangerous rather than how to achieve it. What we don't have is a substantial body of empirical work demonstrating recursive self-improvement in practice.

The closest analogues come from evolutionary computation and neural architecture search, where systems explore spaces of possible configurations to find better-performing variants. These approaches have produced useful results, but they're not recursive in the relevant sense. The search process itself doesn't improve; only the objects being searched over do.

Some recent work on "self-play" in reinforcement learning comes closer. Systems like AlphaZero improve by playing against themselves, and in a loose sense, each iteration of training produces a system that's better at the task. But this is improvement within a fixed framework on a fixed task. The system isn't rewriting its learning algorithms or expanding its domain of competence.

The sample sizes in most RSI-adjacent research are small, and replication has been limited. This isn't a criticism of the researchers involved; these are genuinely hard problems. But it does suggest that the confidence expressed in investor presentations may be outpacing the actual state of knowledge.

Why it matters for robotics

You might wonder why a robotics publication is covering what sounds like an AI capabilities question. The connection is direct. Robotics has always faced a learning problem: physical systems are expensive to train, real-world data is hard to collect, and simulation-to-reality transfer remains imperfect.

If RSI worked, even in limited forms, it could accelerate robotics research dramatically. Imagine a manipulation system that could identify its own failure modes, propose new training scenarios, and improve its grasping capabilities without human intervention. The efficiency gains would be substantial.

But the flip side is also true. Robotics provides a useful reality check on RSI claims. Physical systems impose constraints that pure software doesn't face. A language model can generate millions of candidate improvements and test them quickly. A robot that damages itself trying an untested movement strategy is out of commission for repairs. The feedback loops are slower, the costs of failure are higher, and the evaluation metrics are less forgiving.

Actually, the research shows that robotics has historically been where AI hype goes to die. Capabilities that seem impressive in simulation or in narrow benchmarks often fail to generalise to physical systems operating in unstructured environments. If RSI is real, it will need to work in the physical world eventually. That's a high bar.

Open questions

Several things remain unclear about the current RSI push.

First, we don't know yet what specific technical approaches these new labs are pursuing. The public information is heavy on vision statements and light on methodology. This makes it difficult to evaluate whether the underlying research is sound.

Second, it's too early to say whether the scaling hypothesis extends to self-improvement. Current evidence suggests that larger models are better at many tasks, including tasks related to coding and reasoning. But the jump from "better at coding" to "capable of improving itself recursively" is substantial. There may be phase transitions or barriers that only become apparent at scales we haven't reached.

Third, the evaluation problem I mentioned earlier is, in a way, the crux of the whole endeavor. How would we know if a system had achieved genuine recursive self-improvement? What metrics would distinguish real progress from overfitting to benchmarks? The AI field has a troubled history with evaluation, and RSI seems likely to make these problems worse, not better.

What I'd want to see next

If I were reviewing grant proposals in this space (and I'm not, to be clear, just thinking out loud), I'd want to see several things.

Clear definitions would be a start. What exactly does "self-improvement" mean in the context of a specific system? What's the baseline? What counts as improvement, and how is it measured?

Small-scale demonstrations would help. Before claiming that RSI will transform AI research, show me a system that has improved itself in some measurable way, even a modest one. Show me the before and after. Show me that the improvement generalises beyond the specific evaluation used to guide the process.

Engagement with the theoretical literature would also be valuable. The AI safety community has thought carefully about RSI for years, often from a risk perspective. That work identifies real challenges (the evaluation problem, the stability problem, the alignment problem) that any serious RSI research program needs to address.

And finally, I'd want to see honesty about uncertainty. The history of AI is littered with confident predictions that didn't pan out. RSI may be achievable, or it may be another mirage. The responsible position is to acknowledge that we don't know, and to be skeptical of claims that outpace the evidence.

The bottom line

Recursive self-improvement is a compelling idea with a long history and, currently, limited empirical support. The new wave of RSI-focused companies may be onto something, or they may be riding a hype cycle that will disappoint investors and set back legitimate research.

What we can say is that the goal is proving elusive. The companies pursuing RSI haven't demonstrated it. The theoretical foundations are incomplete. The evaluation challenges are severe. None of this means RSI is impossible. It means that the confident claims being made in pitch decks should be treated with appropriate skepticism.

For robotics specifically, RSI remains a tantalising possibility rather than an imminent reality. Systems that can identify their own weaknesses and suggest improvements would be genuinely useful. But we're not there yet, and the path forward is less clear than the headlines suggest.

I'll be watching this space closely. The research is interesting even when the claims are overblown. And if someone does crack recursive self-improvement, even in a limited domain, it will be worth knowing about. For now, though, I'd recommend a healthy dose of skepticism. We've been here before, and the gap between vision and reality has a way of persisting longer than anyone expects.