Teaching Robots the Old-Fashioned Way Still Works Best, Study Confirms

The Quality Problem Nobody Talks About

The second paper addresses something more interesting to me: demonstration quality. They've built what they call a Data Quality Assessment and Feedback framework, DQAF, that gives operators immediate feedback after each teaching episode.

Here's the problem they're solving. A demonstration can be "successful" in that the robot completes the task, but still be garbage for training purposes. Maybe the operator made a bunch of jerky corrections. Maybe they pushed the arm near its joint limits. Maybe the motion path was inefficient. Binary success/failure doesn't capture any of that.

The system extracts signals like sub-task progress, motion smoothness, stalls, and kinematic limits, then converts them into natural language feedback. Instead of "episode rejected," you get something like "too much time spent correcting near the grasp point" or "elbow joint approached limit three times."

In their pilot study with three novice operators, the one receiving this automated feedback improved faster than those who didn't. Small sample, early days, but the direction makes sense.

I called my old colleague at Siemens about this last week. He's been dealing with the same issue on their end: companies collecting thousands of demonstrations for AI training, then discovering half of them are unusable. The curation process is brutal, basically watching videos at 2x speed and making judgment calls.

What This Means for the Industry

We're in a weird moment for industrial automation. Everyone wants "Physical AI," flexible systems that can adapt to new tasks without months of programming. That requires demonstration data, lots of it, and quality matters more than quantity.

The papers don't say this explicitly, but I'll say it: we're probably going to see a return to kinesthetic teaching for high-value demonstrations, combined with automated quality filtering. Joysticks and VR controllers have their place for remote operation, but for building training datasets? Give me hands-on guidance every time.

What remains unclear is how these findings scale. Eight participants in a lab is one thing. Deploying this in a warehouse with dozens of operators, varying skill levels, and the pressure of production schedules is something else entirely. The DQAF system looks promising, but we don't know yet how operators respond to automated feedback over weeks or months. Do they improve? Do they get frustrated? Do they start gaming the metrics?

I've seen plenty of good ideas from research labs that fell apart on the factory floor. This one, though, actually addresses a real problem I've watched companies struggle with. That's more than I can say for most of what crosses my desk.

Mark Kowalski · 15 hours ago · 4 min