Researchers Keep Finding New Ways to Trick Robot Brains. Should We Be Worried?
Three separate papers this month show how easy it is to hijack vision-language-action models with adversarial patches and poisoned training data. The robots don't even know they're compromised.
画像クレジット: Lottie animation by Centre Robotics (LottieFiles Free, used with credit). · source
What happens when you can make a robot hand someone a knife instead of an apple, and the robot thinks it's doing exactly what you asked?
That's not a hypothetical. It's the result of a new attack called TRAP, one of three papers published this month that expose serious security holes in the AI systems powering the next generation of robots. And honestly, I'm surprised we're not talking about this more.
Let me back up. The robots we're talking about here use something called vision-language-action models, or VLAs. These are the systems that let a robot look at a scene, understand a spoken command like "hand me the apple," and figure out how to actually move its arm to do that. Companies like Physical Intelligence, Google DeepMind, and a bunch of startups are betting big on VLAs as the path to general-purpose robots.
The more advanced versions use chain-of-thought reasoning, basically making the robot "think out loud" about what it's seeing and what it should do. This makes the robots more interpretable and helps them generalize to new situations. Sounds great, right?
Here's the catch: that reasoning process creates a new attack surface.
Researchers from multiple institutions showed that you can hijack a robot's chain-of-thought reasoning with nothing more than an adversarial patch. Think of it as a specially designed image, maybe printed on a tablecloth or stuck to a surface, that messes with the robot's visual processing in very specific ways. The robot sees the patch, its reasoning gets steered toward an adversary-defined behavior, and it executes that behavior while still believing it's following the original instruction.
関連記事
More in Humanoids
A graph diffusion approach to inverse kinematics and an unsupervised motion retargeting framework both dropped this week, and they're more connected than the coverage suggests.
Aisha Patel · 4 hours ago · 8 min
Two new research papers suggest the future of robotics isn't full autonomy — it's figuring out when humans should take over, and when they shouldn't.
Sarah Williams · 19 hours ago · 6 min
This week's arXiv drops tackle the unsexy but essential problem: how do you make humanoid robots actually safe to deploy?
Aisha Patel · Yesterday · 7 min
A wave of new research suggests we can train humanoid robots without expensive human demos. I'm not sure we've thought through what that means.
The scary part? They tested this in the real world. Printed the patch on paper. Put it on a table. The robot got tricked.
The attacks are getting more sophisticated, not less. A separate paper introduces something called SilentDrift, which exploits a fundamental quirk in how VLA models work. Modern systems use "action chunking," where the robot plans several steps ahead and executes them as a sequence. This creates what the researchers call an "intra-chunk visual open-loop," a window where the robot isn't checking its visual input because it's busy executing a pre-planned sequence.
SilentDrift poisons the training data with trajectories that look completely normal but contain tiny, accumulating perturbations. The result: a 93.2% attack success rate with less than 2% of the training data poisoned. The poisoned trajectories are, and I want to be precise here, "visually indistinguishable from successful demonstrations." You can't tell by looking that something's wrong.
A third paper tackles partially observable attacks, which is researcher-speak for "the attacker doesn't need to see everything the robot sees." Previous work assumed an adversary would have full access to the robot's entire execution trajectory, which is unrealistic. This new approach shows you can generate a fixed adversarial patch by observing just a short prefix of the trajectory, then apply it to all subsequent frames. The attack sustains itself over long time horizons.
Key points from the three papers:
Chain-of-thought reasoning, meant to improve interpretability, actually creates new vulnerabilities
Action chunking, meant to improve efficiency, creates blind spots that can be exploited
Attacks can be stealthy (indistinguishable from normal operation) and persistent (sustaining over long task sequences)
Physical-world attacks work, not just simulations
Poisoning rates as low as 2% can achieve over 90% attack success
I initially thought these were edge cases, the kind of adversarial attacks that work in labs but would be hard to pull off in practice. But after reading through the methodologies, I'm less sure. The SilentDrift attack in particular seems like something that could happen accidentally during data collection, not just through malicious intent. If you're scraping demonstration data from multiple sources, how would you even know if some of it was poisoned?
So what do we actually do about this?
This is where I have to be honest: I don't think anyone has a great answer yet. The papers focus on demonstrating the vulnerabilities, not on robust defenses. Some obvious directions include better anomaly detection during training, runtime monitoring of action trajectories for suspicious deviations, and maybe architectural changes that don't rely so heavily on action chunking.
But there's a tension here. The features being exploited (chain-of-thought reasoning, action chunking, delta pose representations) aren't bugs. They're deliberate design choices that make robots more capable. Removing them would mean giving up real performance gains.
You might be wondering whether this matters right now, given that most VLA-powered robots are still in research labs or controlled industrial settings. And tbh, that's a fair point. We're not talking about robots roaming city streets getting hijacked by adversarial billboards.
But the timeline is compressing. Companies are racing to deploy humanoids and mobile manipulators in warehouses, homes, and healthcare settings. The LIBERO benchmark used in the SilentDrift paper is specifically designed to test household manipulation tasks. These aren't abstract research problems anymore.
What worries me most is the gap between the pace of capability development and the pace of security research. VLA models are improving rapidly, with new architectures and training approaches dropping every few weeks. Security analysis is playing catch-up, and the researchers doing this work are a small community.
I should know this better, but I couldn't find any major robotics companies with public bug bounty programs or dedicated adversarial robustness teams. That seems like a problem. When your robot's brain can be hijacked by a tablecloth, maybe security shouldn't be an afterthought.
The researchers behind TRAP put it bluntly: their findings "highlight the urgent need to secure CoT reasoning in VLA systems." I'd extend that to VLA systems in general. We're building robots that can physically interact with the world and with people. The stakes for getting security wrong are higher than a chatbot saying something offensive.
It remains unclear how the major players in humanoid robotics are thinking about these vulnerabilities. I reached out to a few companies for comment but haven't heard back yet. That silence might mean they're taking it seriously and don't want to discuss it publicly. Or it might mean they haven't thought about it much. I honestly can't tell.