Retail robots are failing basic grocery tasks, and researchers finally have the benchmarks to prove it

The researchers fine-tuned several current VLA models using generated trajectories and found that performance remained poor even on what they call "common retail tasks." They don't release specific success rate numbers in the abstract, which is frustrating (I'd love to know exactly how badly these models fail), but the conclusion is clear: we're not close to general-purpose retail robots.

This connects to something I've been thinking about since I spoke with a warehouse automation consultant last month. She pointed out that most robotics demos happen in controlled environments with perfect lighting, known object sets, and plenty of space between items. Real fulfillment centers have none of that. They're cramped, inconsistent, and constantly changing as inventory shifts.

There's a related paper that dropped recently that might offer part of the solution. AFUN, which researchers are calling a step toward an "affordance foundation model," takes a different approach to the problem. Instead of trying to learn end-to-end control, it focuses on understanding where to interact with objects and how to interact with them, predicting both functional masks (the relevant part of an object) and 3D motion curves (the trajectory of interaction).

The results are genuinely impressive. On affordance segmentation, AFUN outperforms baselines by over 23 points on mean gIoU across eight test sets. For contact-point prediction, it achieves hit-rate gains of 12 to 61 percent over the best competing approach. And crucially, the researchers claim it can be deployed for real-world manipulation without fine-tuning for specific robot hardware.

I initially thought these two papers were making competing claims about the state of robot AI. But after reading both more carefully, I think they're actually complementary. RoboBenchMart shows that current end-to-end VLAs don't generalize well to new domains. AFUN suggests that breaking the problem into more structured components (affordance understanding plus motion prediction) might be a more scalable path forward.

The affordance approach has another advantage that's easy to overlook: explainability. When a robot fails at a task, it's much easier to debug if you can see what region it identified as graspable and what motion it planned. End-to-end models are notoriously opaque. You get a success or failure, but understanding why requires extensive analysis.

Of course, there are limitations here that I should be honest about. AFUN's real-world deployment results are preliminary. The paper shows it working on manipulation tasks, but it's unclear how it would perform in the kind of dense retail environments that RoboBenchMart is designed to simulate. And RoboBenchMart itself is a simulation benchmark. Sim-to-real transfer remains one of the hardest problems in robotics.

What does this mean for the companies promising retail automation in the next few years? Honestly, I'm skeptical of aggressive timelines. The benchmarks suggest we need either fundamentally different architectures, much more diverse training data, or (probably) both. The AFUN data pipeline, which converts heterogeneous robot and human demonstration data into a standardized format, points toward one possible path. But building that kind of infrastructure at scale is expensive and slow.

You might be wondering whether this matters outside of retail. I think it does. Retail is just a particularly good test case because the tasks are well-defined, the economic incentive is enormous (labor costs in fulfillment are a huge line item), and the environment is challenging but not dangerous. If robots can't handle grocery shelves, they probably can't handle construction sites or hospitals either.

The researchers behind RoboBenchMart are releasing their full suite, including baseline models, which should help the field iterate faster. That's the kind of infrastructure work that doesn't get as much attention as flashy demos but actually moves the needle. We need more benchmarks that test robots in realistic conditions, not just more impressive videos of robots in labs.

Robert "Bob" Macintosh · 3 hours ago · 3 min