The New Wave of Robot Learning Research Wants to Skip the Robot Part

Then there's MonoDuo, which tackles a genuinely interesting problem: bimanual robots are rare, but single-arm robots are everywhere. So why not use a single arm plus a human collaborator to collect data for both sides of a bimanual task, then synthesize that into training data for a two-armed system? The approach involves teleoperating one arm while a human does the other side, then swapping, then using pose estimation and inpainting to create synthetic bimanual demonstrations.

They report success rates up to 70% on zero-shot deployment to unseen bimanual configurations, and claim that 25 target robot demonstrations can boost performance by 65-70% over training from scratch. That's here. The tasks are reasonable (box lifting, cloth folding, jacket zipping), and the framing is honest about needing some fine-tuning for best results.

Does any of this work in surgery?

Here's where it gets interesting, or concerning, depending on your tolerance for hype in high-stakes domains.

A separate team evaluated imitation learning for surgical assistance, specifically the grab-pull-release motion that a human assistant performs during suturing. They collected 160 teleoperated demonstrations on an open-source robot arm and benchmarked four different imitation learning architectures: ACT, Diffusion Policy, SmolVLA, and π₀.

The results, per their paper, show 50-75% task success under ideal conditions, with depth error as the dominant failure mode. That's not great for surgery! But when they deployed π₀ in an actual surgeon-robot suturing trial, they got a 92% stitch completion rate.

I have questions. What does "stitch completion" mean versus full task success? How many stitches? What happens with the 8% that fail? The paper highlights depth perception and end-effector design as key priorities for clinical translation, which reads to me as "the robot can't see well enough and the gripper isn't right." Fair enough, but those are not small problems.

The finding that π₀ (with its pretrained vision-language backbone) showed better data efficiency and robustness is interesting. It suggests that the foundation model crowd might be onto something, even if the surgical application remains years away from anything resembling deployment.

So what about the foundation models themselves?

Two papers this week take direct aim at evaluating how well these vision-language-action models actually generalize.

Colosseum V2, a simulation benchmark from what appears to be the ManiSkill team, offers 28 tasks across 13 categories and two robot morphologies. They tested ACT and Pi0.5 and found, surprise, limitations in both base performance and generalization. The benchmark is designed for fast GPU-parallelized evaluation, which is nice, and they claim strong correlations between simulation and real-world metrics. Paper here.

The other evaluation paper looks at keypoint imitation learning, which uses visual foundation models to extract keypoints as an intermediate representation. Over 2000 real-world rollouts (that's a lot!), they found keypoint methods achieve 75% success across five tasks, significantly beating an RGB baseline at 47% but only matching S2-diffusion at 73%. The takeaway, per the paper, is that keypoint imitation learning is data-efficient but doesn't outperform alternative representations and inherits whatever limitations exist in the foundation models used for extraction.

Translation: these methods are only as good as the vision models underneath them, and those models have their own problems.

The small model angle

One paper that caught my attention is ProgVLA, a 0.1 billion parameter vision-language-action model designed for tight compute and memory budgets. The approach uses a two-stage compression scheme to handle long multi-modal sequences and trains auxiliary "progress heads" with offline reinforcement learning to give the model an internal estimate of how far along a task it is.

They claim competitive success rates with substantially larger pretrained baselines, and actually exceed them on long-horizon and harder task tiers. The gains are concentrated on long-horizon and multi-object tasks, which is where you'd expect progress-awareness to matter most. Full paper.

This is the kind of work that makes me cautiously optimistic. Not because 0.1B parameters is magic, but because it suggests the field is starting to think about what's actually deployable. A model that runs on real hardware with real constraints is worth more than a 70B parameter monster that needs a data center.

What's actually happening here

Look, I've covered enough tech cycles to know the pattern. First comes the flood of papers showing impressive results in controlled conditions. Then comes the deployment phase where everything breaks. Then comes the quiet decade of actually solving the hard problems.

Robot learning feels like it's somewhere between phase one and phase two. The results are real, the limitations are acknowledged (mostly), and the problems are being stated clearly: depth perception, end-effector design, sim-to-real transfer, generalization under distribution shift.

What I don't see yet is consensus on what the actual path to deployment looks like. Is it foundation models pretrained on internet data? Is it clever data augmentation from human videos? Is it small models that can run on edge hardware? Is it some combination of all three?

The honest answer is we don't know yet, and anyone claiming otherwise is selling something.

But what do I know. If you want to argue, my email's on the about page.

Mark Kowalski · 2 days ago · 4 min