SpatialAnt: Autonomous Zero-Shot Robot Navigation via Active Scene Reconstruction and Visual Anticipation

• URL: http://arxiv.org/abs/2603.26837v1
• Date: Fri, 27 Mar 2026 08:01:40 GMT
• Title: SpatialAnt: Autonomous Zero-Shot Robot Navigation via Active Scene Reconstruction and Visual Anticipation
• Abstract summary: SpatialAnt is a zero-shot navigation framework designed to bridge the gap between imperfect self-reconstructions and robust execution.<n>We show that SpatialAnt significantly outperforms existing zero-shot methods in simulated and real-world environments.
• Score: 45.461768743080604
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision-and-Language Navigation (VLN) has recently benefited from Multimodal Large Language Models (MLLMs), enabling zero-shot navigation. While recent exploration-based zero-shot methods have shown promising results by leveraging global scene priors, they rely on high-quality human-crafted scene reconstructions, which are impractical for real-world robot deployment. When encountering an unseen environment, a robot should build its own priors through pre-exploration. However, these self-built reconstructions are inevitably incomplete and noisy, which severely degrade methods that depend on high-quality scene reconstructions. To address these issues, we propose SpatialAnt, a zero-shot navigation framework designed to bridge the gap between imperfect self-reconstructions and robust execution. SpatialAnt introduces a physical grounding strategy to recover the absolute metric scale for monocular-based reconstructions. Furthermore, rather than treating the noisy self-reconstructed scenes as absolute spatial references, we propose a novel visual anticipation mechanism. This mechanism leverages the noisy point clouds to render future observations, enabling the agent to perform counterfactual reasoning and prune paths that contradict human instructions. Extensive experiments in both simulated and real-world environments demonstrate that SpatialAnt significantly outperforms existing zero-shot methods. We achieve a 66% Success Rate (SR) on R2R-CE and 50.8% SR on RxR-CE benchmarks. Physical deployment on a Hello Robot further confirms the efficiency and efficacy of our framework, achieving a 52% SR in challenging real-world settings.

Related papers

• LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models [48.57715789642472]
  We introduce LIBERO-Recover Benchmark, a large scale benchmark for failure recovery in robotic manipulation.<n>We collect real execution failures from SOTA embodied models and construct 1,000+ scenarios across four recovery levels.<n>We evaluate four core capabilities: spatial understanding, object structure reasoning, interaction understanding, and topological reasoning.
  arXiv  Detail & Related papers  (2026-09-04T14:15:27Z)
• EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal [63.906929035961134]
  We open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments.<n>The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth.<n>Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build context-aware robots.
  arXiv  Detail & Related papers  (2026-07-15T06:02:41Z)
• RoboSnap: One-Shot Real-to-Sim Scene Generation for Generalizable Robot Learning and Evaluation [55.07260698542592]
  We present RoboSnap, a real-to-sim framework that turns a single RGB image into a simulation-ready scene.<n>Experiments on DROID scenes and real-robot tasks show that RoboSnap achieves reliable trajectory replay in the recovered scenes.<n>To further support real-to-sim research, we introduce DROID-Sim, a real-to-sim companion dataset constructed from 564 real-world scenes in DROID.
  arXiv  Detail & Related papers  (2026-07-07T18:19:10Z)
• Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics [21.834006622805678]
  Vision-language-action (VLA) models are gaining attention in robotics, yet their robustness to adversarial attacks remains largely unexplored.<n>We formulate a partially observable threat model, where an adversary can exploit only a short prefix of the trajectory to generate a fixed patch applied to all subsequent frames.<n>First, we localize the patch using the model's attention maps to identify visually critical regions that correspond to the full instruction.<n>Then, we optimize the patch to disrupt the semantic grounding of target objects and increase the curvature of action trajectories.
  arXiv  Detail & Related papers  (2026-06-02T12:19:28Z)
• Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigation [60.07205156194741]
  Embodied navigation requires an agent to map language and visual observations to a stream of spatial actions that drive a real robot through environments it has never seen.<n>We present Uni-LaViRA, a unified agentic architecture that extends the same insight to four task families and to four heterogeneous real robots.
  arXiv  Detail & Related papers  (2026-05-26T18:52:04Z)
• Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Induction [40.686692342584884]
  RoboCortex is a self-evolving framework that enables robots to autonomously induce navigations and cognitive strategies.<n>Our core innovation is an Autonomous Knowledge Induction mechanism that distills multimodal trajectories into a structured Navigation Heuristic Library.<n>We show that RoboCortex consistently outperforms strong baselines in both task success and exploration efficiency.
  arXiv  Detail & Related papers  (2026-05-18T17:52:14Z)
• RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset [48.645870795753105]
  We introduce Robust Autonomous Data Acquisition for Robotics (RADAR)<n>RADAR is a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle.<n>In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-03-12T11:18:52Z)
• Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution [32.93468341343403]
  We introduce Xiaomi-Robotics-0, an advanced vision-language-action (VLA) model optimized for high performance and fast and smooth real-time execution.<n>Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, endowing it with broad and generalizable action-generation capabilities.<n>We evaluate Xiaomi-Robotics-0 extensively in simulation benchmarks and on two challenging real-robot tasks that require precise and dexterous bimanual manipulation.
  arXiv  Detail & Related papers  (2026-02-13T07:30:43Z)
• From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation [35.79160868966466]
  We propose FSD (From Seeing to Doing), a novel vision-language model that generates intermediate representations through spatial relationship reasoning.<n>Our approach combines a hierarchical data pipeline for training with a self-consistency mechanism that aligns spatial coordinates with visual signals.<n>We show that FSD achieves 40.6% success rate in SimplerEnv and 72% success rate across 8 real-world tasks, outperforming the strongest baseline by 30%.
  arXiv  Detail & Related papers  (2025-05-13T13:20:46Z)
• Learning to navigate efficiently and precisely in real environments [14.52507964172957]
  Embodied AI literature focuses on end-to-end agents trained in simulators like Habitat or AI-Thor. In this work we explore end-to-end training of agents in simulation in settings which minimize the sim2real gap.
  arXiv  Detail & Related papers  (2024-01-25T17:50:05Z)
• Neural Scene Representation for Locomotion on Structured Terrain [56.48607865960868]
  We propose a learning-based method to reconstruct the local terrain for a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the estimates the topography in the robot's vicinity. We propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing data coming from the blind spots of the camera arrangement.
  arXiv  Detail & Related papers  (2022-06-16T10:45:17Z)
• Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
  We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform. We produce a closed-loop controller to reactively push objects in a continuous action space. We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
  arXiv  Detail & Related papers  (2021-11-15T18:50:04Z)
• Rapid Exploration for Open-World Navigation with Latent Goal Models [78.45339342966196]
  We describe a robotic learning system for autonomous exploration and navigation in diverse, open-world environments. At the core of our method is a learned latent variable model of distances and actions, along with a non-parametric topological memory of images. We use an information bottleneck to regularize the learned policy, giving us (i) a compact visual representation of goals, (ii) improved generalization capabilities, and (iii) a mechanism for sampling feasible goals for exploration.
  arXiv  Detail & Related papers  (2021-04-12T23:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.