LLMs Are Now Tuning Autonomous Vehicle Behaviour. That Should Probably Worry You.
Researchers want large language models to rewrite the cost functions that govern how self-driving cars move. Bob Macintosh has some thoughts.
By
Seventy percent. That's roughly the share of autonomous vehicle disengagements that, depending on which study you read, trace back to edge cases the motion planner wasn't tuned for. I'll be honest, that number's been rattling around in my head since I came across a new paper out of arXiv that proposes letting a large language model handle the tuning instead of a human engineer.
The paper, from arXiv cs.RO, describes a framework where an LLM reads a structured scenario description or a plain-English user query and then spits out parameters for something called a risk-aware Model Predictive Path Integral controller. MPPI, for those not deep in the motion planning weeds, is a sampling-based approach that evaluates thousands of possible trajectories in parallel and picks the lowest-cost one. The cost function is the whole game. Get those weights wrong and your vehicle either drives like a nervous student or like someone's uncle who thinks turn signals are optional.
The idea here is that instead of a controls engineer spending weeks iterating on those weights, a user just types something like "drive more conservatively in heavy rain" and the LLM translates that into numerical parameters. There's a human-in-the-loop confirmation step before anything gets deployed, which is good. The simulation results apparently show it works as intended. I'm not dismissing that.
But here's the thing.
The Cost Function Is Not a Preference Setting
When I was at Kuka, we had a saying about motion profiles: you don't let the customer write the trajectory. Not because customers are stupid, but because the gap between what someone describes in words and what they actually want the machine to do is enormous, and that gap is where accidents live. We spent years building interfaces that felt like natural language but were actually tightly constrained option sets underneath. The freedom was an illusion, and a deliberate one.
関連記事
More in Autonomy
Three new navigation papers tackle the same ugly problem: robots that trust bad visual information too much. The fix isn't more AI horsepower. It's teaching machines a little epistemic humility.
Mark Kowalski · 3 hours ago · 6 min
Separate research teams tackled GPS-denied exploration from different angles this week, and together they paint a picture of where robot autonomy is actually heading.
Sarah Williams · 3 hours ago · 6 min
Justin Ernest built a captive LP network to back Anthropic, Anduril, and SpaceX without ever raising a traditional venture fund. Sound familiar?
Mark Kowalski · 10 hours ago · 7 min
A pair of fresh arXiv preprints tackle the tension between real-time planning and honest uncertainty in self-driving systems. Neither is a silver bullet, but the ideas are worth examining carefully.
