What Limits Vision-and-Language Navigation ?

• URL: http://arxiv.org/abs/2605.13328v1
• Date: Wed, 13 May 2026 10:41:24 GMT
• Title: What Limits Vision-and-Language Navigation ?
• Abstract summary: StereoNav is a robust Vision-Language-Action framework designed to enhance real-world navigation consistency.<n>Experiments on R2R-CE and RxR-CE demonstrate that StereoNav achieves state-of-the-art egocentric RGB performance.
• Score: 18.423144111320592
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-and-Language Navigation (VLN) is a cornerstone of embodied intelligence. However, current agents often suffer from significant performance degradation when transitioning from simulation to real-world deployment, primarily due to perceptual instability (e.g., lighting variations and motion blur) and under-specified instructions. While existing methods attempt to bridge this gap by scaling up model size and training data, we argue that the bottleneck lies in the lack of robust spatial grounding and cross-domain priors. In this paper, we propose StereoNav, a robust Vision-Language-Action framework designed to enhance real-world navigation consistency. To address the inherent gap between synthetic training and physical execution, we introduce Target-Location Priors as a persistent bridge. These priors provide stable visual guidance that remains invariant across domains, effectively grounding the agent even when instructions are vague. Furthermore, to mitigate visual disturbances like motion blur and illumination shifts, StereoNav leverages stereo vision to construct a unified representation of semantics and geometry, enabling precise action prediction through enhanced depth awareness. Extensive experiments on R2R-CE and RxR-CE demonstrate that StereoNav achieves state-of-the-art egocentric RGB performance, with SR and SPL scores of 81.1% and 68.3%, and 67.5% and 52.0%, respectively, while using significantly fewer parameters and less training data than prior scaling-based approaches. More importantly, real-world robotic deployments confirm that StereoNav substantially improves navigation reliability in complex, unstructured environments. Project page: https://yunheng-wang.github.io/stereonav-public.github.io.

Related papers

• LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation [52.91080323083409]
  Embodied navigation requires agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and robot embodiments.<n>Here we present LightNav-0, a compact generalist embodied navigation model that elicits the spatial intelligence of a pretrained VLM and aligns it with navigation, without task-specific prediction heads.
  arXiv  Detail & Related papers  (2026-08-31T15:08:53Z)
• Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation [68.02899606570223]
  We propose TAMP-Nav, a unified framework for efficient embodied navigation.<n>First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual coordinates.<n>Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize)
  arXiv  Detail & Related papers  (2026-08-18T08:37:08Z)
• Towards Dual-Brain Minimal Sufficient Representation for Vision-Language Navigation [32.00869718735648]
  BrainNav is a navigation framework grounded in the Principle of Minimal Sufficiency.<n>BrainNav consists of three components: a Logical Anchor Model that implements instruction-aware selective perception, a Minimalist Constraint Alignment module, and a Compression World Model.<n> Experiments show that BrainNav improves over prior SOTA by 2.0 % / 1.0 in SR/SPL on R2R-CE val-unseen and 0.94 % / 0.78 on RxR-CE val-unseen.
  arXiv  Detail & Related papers  (2026-07-25T12:26:47Z)
• UNSEEN: Uncertainty-aware Navigation via Sparse Estimation in Unknown Environments [5.893948403053389]
  Vision-only navigation offers a lightweight alternative, but its performance degrades severely under motion blur, low texture, and illumination changes.<n>We present UNSEEN, a unified uncertainty- and perception-aware navigation framework that explicitly couples localization, mapping, and planning using only a front-mounted camera.
  arXiv  Detail & Related papers  (2026-06-18T07:54:07Z)
• AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild [62.47761809929869]
  Vision-language navigation (VLN) requires intelligent agents to navigate environments by interpreting linguistic instructions alongside visual observations.<n>Current VLN research for unmanned aerial vehicles (UAVs) relies on detailed, pre-specified instructions to guide the UAV along predetermined routes.<n>We propose AutoFly, an end-to-end Vision-Language-Action model for autonomous UAV navigation.
  arXiv  Detail & Related papers  (2026-02-10T11:08:07Z)
• Learning to Tune Like an Expert: Interpretable and Scene-Aware Navigation via MLLM Reasoning and CVAE-Based Adaptation [12.561993540768729]
  We present LE-Nav, an interpretable and scene-aware navigation framework for service robots.<n>To achieve zero-shot scene understanding, we utilize one-shot exemplars and chain-of-thought prompting strategies.<n>Experiments show that LE-Nav can generate hyperparameters achieving human-level tuning across diverse planners and scenarios.
  arXiv  Detail & Related papers  (2025-07-15T05:37:24Z)
• SEMNAV: A Semantic Segmentation-Driven Approach to Visual Semantic Navigation [1.2582887633807602]
  Visual Semantic Navigation (VSN) is a fundamental problem in robotics, where an agent must navigate toward a target object in an unknown environment.<n>Most state-of-the-art VSN models are trained in simulation environments, where rendered scenes of the real world are used, at best.<n>We propose SEMNAV, a novel approach that leverages semantic segmentation as the main visual input representation of the environment.
  arXiv  Detail & Related papers  (2025-06-02T08:19:41Z)
• DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigation [55.888688171010365]
  DORAEMON is a cognitive-inspired framework consisting of Ventral and Dorsal Streams that mimics human navigation capabilities.<n>We evaluate DORAEMON on the HM3D, MP3D and GOAT datasets, where it achieves state-of-the-art performance on both success rate (SR) and success weighted by path length (SPL) metrics.
  arXiv  Detail & Related papers  (2025-05-28T04:46:13Z)
• Ground-level Viewpoint Vision-and-Language Navigation in Continuous Environments [10.953629652228024]
  Vision-and-Language Navigation (VLN) agents associate time-sequenced visual observations with corresponding instructions to make decisions.<n>In this paper, we address the mismatch between human-centric instructions and quadruped robots with a low-height field of view.<n>We propose a Ground-level Viewpoint Navigation (GVNav) approach to mitigate this issue.
  arXiv  Detail & Related papers  (2025-02-26T10:30:40Z)
• NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning [97.88246428240872]
  Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions.<n>Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability.<n>This paper introduces a novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision.
  arXiv  Detail & Related papers  (2024-03-12T07:27:02Z)
• Learning Navigational Visual Representations with Semantic Map Supervision [85.91625020847358]
  We propose a navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps. Ego$2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation.
  arXiv  Detail & Related papers  (2023-07-23T14:01:05Z)
• CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation [73.78984332354636]
  CorNav is a novel zero-shot framework for vision-and-language navigation. It incorporates environmental feedback for refining future plans and adjusting its actions. It consistently outperforms all baselines in a zero-shot multi-task setting.
  arXiv  Detail & Related papers  (2023-06-17T11:44:04Z)
• Learning Synthetic to Real Transfer for Localization and Navigational Tasks [7.019683407682642]
  Navigation is at the crossroad of multiple disciplines, it combines notions of computer vision, robotics and control. This work aimed at creating, in a simulation, a navigation pipeline whose transfer to the real world could be done with as few efforts as possible. To design the navigation pipeline four main challenges arise; environment, localization, navigation and planning.
  arXiv  Detail & Related papers  (2020-11-20T08:37:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.