The VLA Arms Race Is Here, and It Looks a Lot Like 2016 All Over Again

Here's where things get interesting, and by interesting I mean concerning. These models have a nasty habit of forgetting what they learned when you teach them something new. It's called catastrophic forgetting, and it's been a thorn in neural network research for decades.

A team working on something called PHASER (arXiv) is specifically tackling this. Their approach treats robot tasks as having distinct phases, like the different movements in picking something up, and makes sure the model's memory doesn't shortchange any of them. They're reporting a 31% improvement in success rate over standard approaches. That's meaningful if it holds up.

But what do I know. I've watched plenty of promising techniques in other domains turn out to be benchmark-specific tricks that don't generalize. The real test is whether these methods work when you're not carefully controlling every variable.

The benchmarking problem

Speaking of benchmarks, there's a new framework called VLA-Arena (arXiv) that's trying to bring some rigor to how we evaluate these systems. They've designed 170 tasks across different difficulty levels, and they're finding that current VLAs have, and I quote their findings here, "a strong tendency toward memorization over generalization."

That's a polite way of saying these models are great at doing exactly what they've seen before and terrible at doing anything slightly different. The benchmark also tests for safety constraints, and the results there are not encouraging. These robots don't naturally avoid doing dangerous things unless you've specifically trained them not to do those exact dangerous things.

I've been covering tech long enough to know that benchmarks get gamed. Give researchers a target and they'll optimize for it, sometimes at the expense of actual capability. But at least someone's trying to measure this stuff systematically.

The hardware question everyone's ignoring

Most of these papers are simulation-only, or they test on one or two carefully controlled real-world setups. A group releasing something called OpenEAI-Platform (arXiv) is at least being honest about the hardware problem. They've built a low-cost 6-degree-of-freedom arm and open-sourced the whole thing, mechanical designs and all.

Their VLA model, trained only on open-source datasets, apparently matches the performance of systems trained on much larger proprietary data. That's interesting if true, though I'd want to see independent replication. The open-source angle matters because right now, basically, the only way to do this research is to either have a lot of money or to work at one of a handful of well-funded labs.

There's also work on edge deployment, running these models on limited hardware rather than cloud servers. A system called SCOPE (arXiv) is designed to run entirely on local compute for controlling pan-tilt-zoom cameras. They're finding that the planning model and the perception model both matter, and that smaller mixture-of-experts architectures can match bigger dense networks at lower latency. This is the kind of practical engineering work that actually matters for real deployment.

The safety angle

One paper caught my eye specifically because it's not about making robots smarter, it's about making them safer after they're already trained. The approach is called Latent Activation Editing (arXiv), and it's borrowed from techniques people use to steer large language models away from saying harmful things.

The idea is you monitor the robot's internal activations during operation, detect when it's about to do something dangerous (like fly a drone into an obstacle), and nudge those activations toward safer behavior without retraining the whole model. They're claiming nearly 90% fewer collisions in multi-drone navigation scenarios.

This is the kind of work that doesn't get as much attention as the flashy capability improvements, but it's arguably more important. A robot that's 96% successful but occasionally catastrophically fails is not actually that useful. We learned this lesson with self-driving cars, though apparently we need to learn it again.

What I think is actually happening

Look, I've watched enough technology hype cycles to be skeptical of any claim that we're on the verge of general-purpose robots. We're not. These systems are still brittle, they still fail in unexpected ways, and they still require enormous amounts of carefully curated training data.

But something is different this time. The foundation models are genuinely capable in ways that previous approaches weren't. The ability to leverage pretrained vision and language understanding means you're not starting from scratch every time. And the open-source movement is real, with multiple groups releasing hardware designs, training code, and datasets.

The question is whether this translates into actual deployed systems that do useful work, or whether it remains an academic exercise for another decade. I genuinely don't know. The papers are impressive, the benchmarks are improving, but the gap between "works in the lab" and "works in a warehouse" remains enormous.

One thing I'm confident about: we're going to see a lot more VLA papers in the coming months. Whether any of them lead to robots you'd actually want in your home or factory, well. That remains unclear. If you want to argue about it, my email's on the about page.

Mark Kowalski · 11 hours ago · 5 min