VLAs Can't See in 3D, and Two New Papers Finally Quantify the Problem

But VLAs are appealing because they promise generalization. Train once, deploy anywhere. The problem is they're trying to learn physics and geometry implicitly, from 2D images and language, when we already have explicit methods that work. It's like teaching someone to catch a ball by describing trajectories in words instead of just letting them practice.

The first paper's team tested three different architectures for injecting geometric understanding into VLAs, keeping implementation details as similar as possible for fair comparison. That's good methodology. They also looked at training data volume, number of cameras, and reconstruction quality. The details matter here, and I'm glad someone's being rigorous about it.

The 3DVLA framework from the second paper takes a different approach: a "plug-and-play" system that bolts 3D reasoning onto existing VLAs without requiring expensive instance-level annotations. They tested on LIBERO-Plus and RoboTwin 2.0 benchmarks and claim "consistent and significant gains." I'd want to see those numbers in real-world conditions, but the direction seems right.

What happens next

I called my old colleague at Fanuc last week (he's still there, somehow), and his take was that the industrial side isn't worried about VLAs yet. They're solving different problems. Factory robots need repeatability and speed. VLAs are chasing flexibility and generalization.

But that gap is going to close. The question is whether it closes by VLAs getting better at geometry, or by someone building a hybrid that uses proper 3D sensing where it matters and language models for task understanding.

My money's on the hybrid approach. We don't need robots that can infer depth from monocular images when we can just give them a depth sensor. What we need is systems smart enough to know when their geometric understanding is insufficient. That remains unclear from these papers, whether these improved VLAs know what they don't know.

The "geometric gap" framing is useful though. It gives us something to measure against. In a few years we'll be able to track whether that gap is shrinking, and by how much. That's progress, even if the robots still can't see in 3D quite yet.

James Chen · 7 hours ago · 4 min