The New Wave of Self-Improving Robots Sounds Familiar (Because It Is)

SOLE-R1, from a team at MIT based on their project page, takes a different angle. They built a video-language reasoning model that serves as the sole reward signal for reinforcement learning. The robot watches video of its own actions, reasons about progress using chain-of-thought, and generates its own rewards. No ground-truth rewards, no success indicators, no demonstrations. They tested it on 24 unseen tasks across four simulation environments and a real robot, and it substantially outperformed other approaches including, they claim, GPT-5 and Gemini-3-Pro as reward models.

LACY, from arXiv, introduces bidirectional learning: the robot learns to generate actions from language AND explain its actions back in language. The idea is that a robot that can explain itself develops richer internal representations. They report a 56.46% average improvement in task success rates.

Finally, OHP-RL from arXiv keeps humans in the loop but treats their interventions as preference information rather than demonstrations to imitate. A "preference gate" decides when and how much human guidance should influence learning. Tested on three real-world manipulation tasks with a Franka robot, it achieved strong success rates with substantially lower human intervention effort.

What's actually new here

Call me old-fashioned, but I've been covering tech long enough to know that academic benchmarks don't always translate to real-world performance. The LIBERO benchmark is useful! But it's still a benchmark. The gap between "works in simulation" and "works in your warehouse" remains enormous, and these papers are mostly honest about that, which I appreciate.

That said, there's a through-line in this research that feels genuinely different from the last few hype cycles.

The old approach to robot learning was essentially supervised: collect demonstrations, train a model, deploy it, hope it generalizes. When it didn't generalize (and it usually didn't), you collected more demonstrations. This was expensive and slow, and it's why most industrial robots still run on hand-coded routines.

What these papers are proposing, in various ways, is closing the loop. The robot tries something, evaluates its own performance, adjusts its own learning objectives, and tries again. Some of them use language models to generate rewards or explanations. Some of them use humans as preference oracles rather than demonstration sources. Some of them restructure their own feature representations.

This is the self-driving car hype cycle all over again, in the sense that we're once again being told that the system will improve itself through experience. But there's a key difference: these systems are designed for manipulation tasks in constrained environments, not open-world driving. The problem is smaller. The feedback loops are tighter. The stakes are lower.

The parts that worry me

Here's what none of these papers adequately address, at least not to my satisfaction.

First, reward hacking. SOLE-R1 claims to be "markedly more robust to reward hacking" than alternatives, but robust is a relative term. When you let a system generate its own rewards, you're trusting that its internal model of success aligns with your external model of success. Sometimes it does! Sometimes the robot learns to make the reward signal go up without actually completing the task. The paper acknowledges this is a problem but the evidence that they've solved it is, well, limited.

Second, distribution shift. Agentic-VLA reports impressive numbers on cross-task transfer, but 31.2% success without task-specific demonstrations still means 68.8% failure. That's fine for a research benchmark. It's not fine for a production system. The paper is honest about this being a "significant step toward" adaptive systems rather than the destination, but I've seen a lot of significant steps that didn't lead anywhere.

Third, we don't know yet how these approaches interact with each other. Each paper proposes a different mechanism for self-improvement. Some of them might be complementary. Some of them might be redundant. Some of them might actively conflict. The field is throwing ideas at the wall, which is appropriate for research, but it makes it hard to know which ideas will survive contact with reality.

What this means for the industry

If you're running a robotics company, here's my read on what's actually actionable.

The short-term impact is approximately zero. These are research papers, not products. The techniques need years of engineering work before they're deployable at scale, and that's assuming they work as advertised.

The medium-term impact is more interesting. If even some of these self-improvement mechanisms pan out, the economics of robot deployment change significantly. Right now, the bottleneck is data collection and fine-tuning for every new task and environment. If robots can adapt themselves with minimal human input, the deployment cost drops and the addressable market expands.

The long-term impact, I genuinely don't know. We've been promised autonomous robot learning for decades. The techniques keep getting better. The benchmarks keep improving. And yet most robots in the wild still run on explicit programming. Something has to give eventually, but what do I know.

The historical parallel that actually matters

Here's the comparison I keep coming back to.

In the early 2010s, deep learning started crushing benchmarks in computer vision. ImageNet accuracy went from 74% to 85% to 95% in just a few years. The hype was enormous. Everyone was going to have self-driving cars and robot butlers by 2020.

That didn't happen, obviously. But what DID happen was that deep learning quietly took over every vision system in production. It just took longer than the hype suggested, and the applications were different than predicted. We got better photo search and face unlock and warehouse sorting, not robot butlers.

I think we're in a similar moment with robot learning. The benchmarks are improving faster than the deployments, which creates a hype gap. The young founders raising money on these papers are going to struggle when reality sets in. But the underlying techniques are real, and they'll show up in production eventually, probably in ways we're not predicting.

So yes, I'm skeptical of the timeline. I'm skeptical of the claims. I'm especially skeptical of anyone telling you this is a paradigm shift or a game-changer (those are words people use when they don't have specifics).

But I'm not skeptical of the direction. Robots that improve themselves, even slowly, even imperfectly, are better than robots that don't. And this batch of research, for all its limitations, represents genuine progress on that front.

If you want to argue about it, my email's on the about page.

Sarah Williams · 1 hour ago · 4 min