LLMs Are Now Tuning Autonomous Vehicle Behaviour. That Should Probably Worry You.
Researchers want large language models to rewrite the cost functions that govern how self-driving cars move. Bob Macintosh has some thoughts.
By
Seventy percent. That's roughly the share of autonomous vehicle disengagements that, depending on which study you read, trace back to edge cases the motion planner wasn't tuned for. I'll be honest, that number's been rattling around in my head since I came across a new paper out of arXiv that proposes letting a large language model handle the tuning instead of a human engineer.
The paper, from arXiv cs.RO, describes a framework where an LLM reads a structured scenario description or a plain-English user query and then spits out parameters for something called a risk-aware Model Predictive Path Integral controller. MPPI, for those not deep in the motion planning weeds, is a sampling-based approach that evaluates thousands of possible trajectories in parallel and picks the lowest-cost one. The cost function is the whole game. Get those weights wrong and your vehicle either drives like a nervous student or like someone's uncle who thinks turn signals are optional.
The idea here is that instead of a controls engineer spending weeks iterating on those weights, a user just types something like "drive more conservatively in heavy rain" and the LLM translates that into numerical parameters. There's a human-in-the-loop confirmation step before anything gets deployed, which is good. The simulation results apparently show it works as intended. I'm not dismissing that.
But here's the thing.
The Cost Function Is Not a Preference Setting
When I was at Kuka, we had a saying about motion profiles: you don't let the customer write the trajectory. Not because customers are stupid, but because the gap between what someone describes in words and what they actually want the machine to do is enormous, and that gap is where accidents live. We spent years building interfaces that felt like natural language but were actually tightly constrained option sets underneath. The freedom was an illusion, and a deliberate one.
関連記事
More in Autonomy
A startup called REO says it will sell a pickup truck for $21,500. The price is striking. The evidence for it is less so.
Aisha Patel · 24 Jun · 9 min
Researchers are patching the 'trajectory scoring gap' in sidewalk robots with VLMs and human attention modeling. The ideas are clever. The caveats are real.
Mark Kowalski · 20 Jun · 6 min
Two new papers tackle one of robotics' most stubborn problems: getting a robot to figure out its location using LiDAR, without needing to have visited the place before.
Sarah Williams · 19 Jun · 5 min
The defense tech startup is moving from drones to full autonomous fighters, and it raises questions about where the line between AI autonomy and human oversight actually sits.
