The robots are getting smarter, but are we checking their work?

Even on simpler tabletop tasks, the high-performing baselines still showed 13 to 15 percent unsafe episode rates. That's not nothing. That's one in seven.

So what

Look, I called my old colleague at Siemens last week about something unrelated, and we ended up talking about this exact problem. His take was basically: we've spent decades building safety into the control layer, and now we're bolting AI on top that operates in a completely different paradigm.

The literature review I mentioned earlier lays out the problem pretty clearly. These foundation models can issue what they call "silent failures." The model looks confident. The output seems plausible. It's semantically aligned with the instruction. But the physical assumption underneath is wrong, and nothing catches it before the arm moves.

Sensor drift. Occlusion. Distribution shift. Hallucinated affordances (which is a fancy way of saying the model thinks it can do something it can't). The researchers went through robotics simulation, embodied safety benchmarks, safe control methods, runtime assurance, all of it. Their conclusion: no single approach currently provides a complete runtime authorization boundary between the AI model and physical execution.

That's a polite way of saying we're flying a bit blind here.

What happens next

There's some interesting work on what they're calling belief-space safety filters. One paper proposes using conformal prediction to certify high-probability safety while the robot is actually running. The idea is that you verify the safety filter in regions where the robot's inference is expected to be reliable.

It's clever. Whether it's practical at scale, I genuinely don't know yet. The researchers tested it in a simulated human-vehicle interaction benchmark, which is a long way from a warehouse floor.

I keep thinking about the Fanuc M-20iA we had running pick-and-place back in 2015. If something went wrong, we knew exactly what went wrong. The error code told us. The safety system was deterministic. I could trace the logic.

With these new systems, the model might make a decision that's, sort of, reasonable based on its training, but completely inappropriate for the actual situation. And the scary part is it won't necessarily look like an error. It'll look like the robot doing its job.

The industry's going to have to figure this out. We can't just keep measuring task success and calling it done. The SafeVLA-Bench folks are right to separate success from safety in their metrics. But metrics are one thing. Actually deploying guardrails that work in production, with real noise and real edge cases, that's something else entirely.

I don't have the answer. I'm not sure anyone does yet. But at least people are asking the right questions now.

Robert "Bob" Macintosh · 2 hours ago · 3 min