2 Billion Grasps: The Brute Force Approach to Teaching Robots New Hands

The key insight there is using "affordance correspondence," basically finding meaningful keypoints that transfer across different object geometries. If you know where to grab a mug handle, that knowledge should transfer to grabbing a pitcher handle.

They're claiming zero-shot generalization to "truly unseen objects," which is ambitious. In my experience, "unseen" in lab demos and "unseen" in a real warehouse are different beasts. But the approach is sound, and combining semantic understanding with end-to-end learning is probably the right direction.

So what

Look, I've been skeptical of the "just add more data" approach to robotics. When I started in this industry, the thinking was that you needed careful mechanical design, precise calibration, deterministic motion planning. The idea that you could train your way out of hardware variation seemed, I don't know, sloppy.

But two billion grasps is hard to argue with. If you can generate that much synthetic data cheaply (and you can, compute is cheap now), and if it actually transfers to real hardware (still the big if), then maybe brute force is the answer.

What remains unclear is how these approaches handle edge cases. Industrial grasping isn't just about picking up random objects, it's about picking up the same object ten thousand times without failure. The papers don't really address cycle time, reliability over long runs, or what happens when the lighting changes or there's oil on the parts.

What happens next

I expect we'll see more work in this direction. The combination of generative models for data synthesis and diffusion models for grasp planning is powerful. Toyota Research Institute being involved suggests there's real money behind commercializing this.

For integrators, the promise is obvious: buy a new gripper, plug it in, and have it work without months of tuning. We're not there yet. But we're closer than we were when I retired from Kuka, and that's sort of remarkable when I think about it.

The procedural gripper generation is particularly interesting to me. Back in the day, we'd design grippers by hand, test them physically, iterate slowly. Now you can generate thousands of gripper designs, simulate their grasping performance, and train a model that handles all of them. It's a different world.

Whether that world actually works better for manufacturing, well, ask me in five years. I've seen enough "revolutionary" robotics research that never made it past the demo stage. But the scale of these datasets, the quality of the simulation, the transfer results... I'm more optimistic than usual. Which for me means about 40% confident instead of my baseline 20%.

Robert "Bob" Macintosh · 2 hours ago · 4 min