Six Papers, One Message: VLA Models Still Can't Handle Long Tasks Without Help

The failure detection problem

Here's what concerns me more than raw success rates: we still can't reliably tell when these systems are failing.

A paper called "Hide-and-Seek" tackles VLA failure detection as a learning problem. The core issue is that trajectory-level labels (did the whole task succeed or fail?) don't tell you which specific action caused the problem. Their solution uses contrastive learning to localize failure signals without step-level annotation.

This is clever work. It's also an admission that we're deploying systems we can't monitor properly. The paper evaluates on three VLA policies, OpenVLA, π₀, and π₀.₅, and achieves "state-of-the-art" failure detection. But state-of-the-art in a field this young is a low bar.

Memory remains unsolved

Look, the most fundamental limitation might be the simplest one. Current VLAs are, as one paper puts it, "primarily stateless." They struggle with tasks that require remembering what happened earlier in the sequence.

The "Notes-to-Self" paper proposes giving VLAs a language scratchpad, essentially a place to write down what they've seen and what they're planning to do. It's a reasonable hack. The authors test on memory-dependent tasks and show significant improvements for both recurrent and non-recurrent models.

But this feels like we're reinventing working memory from scratch. The fact that we need explicit scratchpads to remember object positions suggests these models aren't learning the kind of representations that would generalize to novel memory-dependent scenarios. It remains unclear whether this approach scales beyond carefully constructed benchmarks.

The scaffolding pattern

SPR (See, Plan, Rewind) adds explicit milestone tracking and failure recovery. ELAN4D adds 4D supervision through robot keypoint prediction. MPVI adds motion planning. Notes-to-Self adds a scratchpad.

Every one of these papers improves on baselines. Every one does it by adding structure that the base VLA model lacks. The common thread is that pure end-to-end learning isn't enough.

This isn't necessarily bad news. From my time in hardware engineering, I learned that the best systems are often hybrids. You use learning where it excels (perception, semantic understanding) and classical methods where they're more reliable (planning, failure recovery). The MPVI paper's 113% improvement comes from exactly this kind of integration.

But it does suggest that the "scale solves everything" hypothesis has limits. These papers are finding those limits and, sort of, routing around them.

What's missing from the research

I only found limited real-world evaluation across these papers. ELAN4D and Notes-to-Self include some physical robot experiments, but the bulk of the results are simulation. That's an ambitious gap to bridge.

The other thing I'd want to see: how do these methods compose? If you add milestone tracking AND mixture of horizons AND a scratchpad, do the benefits stack? Or do they interfere? Nobody's tested that yet, as far as I can tell.

The field is moving fast. Six papers in roughly the same problem space, all finding different ways to prop up VLA reliability. The 99% success rate is real, under the right conditions. The question is whether we can get those conditions outside the lab.

Sarah Williams · 3 hours ago · 6 min