Visual Navigation Is Getting a Geometry Upgrade, Whether It Needs One or Not

Think of it like this: you don't need a GPS coordinate for your couch to know that if you walk forward from the doorway, you'll hit it. You just need to understand the relationship between where you are and where the couch is, relative to your current view.

They construct a map from image sequences by establishing pixel correspondences between image pairs, then use that pixel-level graph for path planning. The result is what they call a "WayPixel Costmap," which sounds like something a kid would name but actually represents a pretty dense representation of navigable space. They train a controller on this costmap to predict trajectories.

The key claims:

• Geometrically accurate without requiring global consistency
• Works better than image-level or object-level conditioning for control prediction
• Validated in simulation and real-world demos across four navigation task types

I haven't seen the actual performance numbers beyond what's in the abstract, so take the "highly capable" self-assessment with appropriate skepticism. But the core idea, that you can get geometric accuracy without the brittleness of global maps, is sound. I've seen this movie before with SLAM systems that promised robustness and delivered it about 60% of the time. We'll see if this approach fares better.

Scene graphs that actually adapt

FOUND-IT takes a different angle. Instead of pixel-level connectivity, they're building hierarchical 3D scene graphs, basically a structured representation of objects, regions, and their relationships. The twist is that the granularity adjusts based on the task.

This is actually important! If your robot is navigating across a room, it doesn't need to know about individual knobs on the stove. But if it's trying to turn on a burner, suddenly those knobs matter a lot. FOUND-IT claims to handle this dynamically, adjusting representation granularity as tasks evolve.

They're using geometric foundation models (VGGT specifically gets a mention) to estimate things like bounding boxes, and they've added a head to these models to directly reconstruct traversability information. The "places" layer of the scene graph, basically where the robot can and can't go, comes out of the same foundation model pipeline.

Some specifics that stood out:

• 79% higher accuracy on the ASHiTA SG3D task grounding benchmark (compared to what baseline, I'm not sure, the abstract doesn't say)
• Runs in real-time on a Jetson Thor, which is notable because embedded compute is always the bottleneck
• Works on casually captured YouTube videos of apartment tours, which suggests reasonable robustness to messy input
• Handles "loco-manipulation" tasks where navigation and manipulation interleave

The YouTube apartment tour demo is actually the most interesting validation to me. Academic robotics papers love to show results on carefully captured datasets with perfect lighting and no motion blur. Realtor videos are, shall we say, not that. If FOUND-IT actually works on that kind of input, it's a meaningful step toward real-world deployment.

What this means, probably

Look, I've been covering tech since the 90s, and robotics specifically for longer than I care to admit. I've watched navigation go from "we need perfect maps" to "maps are dead, long live learning" to "okay maybe some geometric structure is useful after all." This is the self-driving car hype cycle all over again, just playing out in indoor robotics with a 5-year delay.

Both of these papers are pointing in the same direction: foundation models have gotten good enough at extracting geometric information from images that we can build useful spatial representations without the old SLAM infrastructure. That's genuinely new. The question is whether these representations are robust enough for deployment or just good enough for papers.

I remain skeptical that pixel-level anything will scale gracefully to large environments, the memory and compute costs tend to sneak up on you. And dynamic granularity in scene graphs sounds great until you're debugging why your robot decided the entire kitchen was a single object during a manipulation task.

But what do I know. These are smart people working on hard problems, and the trend toward foundation-model-first perception is clearly where the field is heading. If you're building navigation systems today, you should probably be paying attention to this stuff.

The code for FOUND-IT is supposedly coming upon publication. MASt3R-Nav's availability isn't mentioned. If you want to argue about any of this, my email's on the about page.

Aisha Patel · 3 hours ago · 7 min