Drone Swarms Are Getting Smarter, But Let's Talk About What That Actually Means

The second paper takes a different approach entirely. This one, from researchers working on multi-agent reinforcement learning, goes end-to-end: raw LiDAR data in, control commands out, no hand-crafted features in between. They've built what they call a "Predictive Spatio-Temporal Observation" (PSTO) that basically creates a unified picture of obstacles, targets, and teammates in a single representation.

Now, I called my old colleague at Siemens who's been following the perception side of autonomous systems more closely than I have, and he made a point that I think is important. The fact that they're working directly from LiDAR rather than relying on "privileged" ground-truth state information (basically, perfect knowledge that you'd never have in the real world) is significant. A lot of the impressive-looking swarm demos you see online are essentially cheating, they know exactly where everything is because the simulation tells them. This work, at least according to the arXiv paper, doesn't do that.

They also claim the policy scales to different team sizes without retraining, which, if true, is actually kind of remarkable. In my experience with multi-robot systems, you usually have to retune everything when you change the number of agents.

But here's where I have to pump the brakes a bit. The LAEI paper is simulation-only, as far as I can tell. The second paper does have outdoor experiments with actual quadrotors, which is encouraging, but "fully autonomous outdoor experiments" can mean a lot of things. I don't know how cluttered the environment was, how fast the targets were moving, or what the failure rate looked like. The paper says they achieved "superior capture efficiency," but compared to what baseline? Other learning-based methods, sure. But what about a skilled human operator? What about a simpler rule-based system that's been tuned for the specific environment?

I'm not trying to be a curmudgeon here. This is real progress. The fact that we're seeing end-to-end learned policies that work on actual hardware, outdoors, with only onboard sensing, that's genuinely new. Five years ago this would have been a pipe dream.

Where I think this is heading is toward hybrid systems that look a lot like what we built for ground-based automation, but with the added complexity of three dimensions and much faster dynamics. You'll have local autonomy for the stuff that needs to happen fast (collision avoidance, basic navigation), layered supervision for mission-level coordination, and probably some human oversight for the foreseeable future.

The military applications are obvious, and I suspect that's where a lot of the funding is coming from, though neither paper says so explicitly. But the civilian applications are interesting too. Search and rescue, infrastructure inspection, agricultural monitoring. Anywhere you need persistent coverage of a large area with multiple vehicles.

It's too early to say whether these specific approaches will be the ones that get deployed at scale. But the direction of travel seems clear. Swarms are going to get more capable, more resilient, and more autonomous. Whether we're ready for that, well, that's a different question entirely.

Robert "Bob" Macintosh · 18 Jun · 4 min