P2DNav: Panorama-to-Downview Reasoning for Zero-shot Vision-and-Language Navigation

• URL: http://arxiv.org/abs/2605.19634v1
• Date: Tue, 19 May 2026 10:18:46 GMT
• Title: P2DNav: Panorama-to-Downview Reasoning for Zero-shot Vision-and-Language Navigation
• Abstract summary: P2DNav is a hierarchical framework for zero-shot vision-and-language navigation.<n>P2DNav consists of three core components: Panorama-to-Downview (P2D), Sliding-Window Dialogue Memory (SDM), and Reflective Reorientation Mechanism (RRM)
• Score: 30.45812977392826
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-and-language navigation (VLN) requires an embodied agent to ground natural-language instructions into executable navigation actions in unseen environments. Existing zero-shot methods typically rely on additional waypoint prediction modules, which often entangle high-level directional reasoning with fine-grained local grounding, leading to error-prone and unstable decisions. In this paper, we propose P2DNav, a hierarchical framework for zero-shot vision-and-language navigation. P2DNav consists of three core components: Panorama-to-Downview (P2D), Sliding-Window Dialogue Memory (SDM), and Reflective Reorientation Mechanism (RRM). P2D explicitly decomposes navigation decision-making into two stages: panoramic direction selection and downview local grounding. It first selects the instruction-relevant direction from a 360° panorama, and then predicts a pixel-level target point from the downview RGB observation in that direction. In addition, SDM organizes navigation history as a multi-turn dialogue context and maintains recent visual observations within a sliding window to support long-horizon navigation. RRM further enables reflective reorientation by assessing the reliability of local grounding based on the downview observation and returning to panoramic direction selection when necessary. Experiments on the R2R-CE benchmark show that P2DNav achieves strong performance among zero-shot methods. In particular, compared with the state-of-the-art (SOTA) zero-shot waypoint-based and waypoint-free methods, P2DNav achieves SR gains of 146.6% and 58.9%, respectively, demonstrating the effectiveness of P2D, SDM, and RRM for zero-shot VLN. Code will be released for public use.

Related papers

• One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints [10.32396390031609]
  Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to navigate unseen environments by following natural language instructions.<n>We propose O2C-Nav, an efficient zero-shot navigation framework that calls only a single large model once per decision step.<n>Our approach introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers.
  arXiv  Detail & Related papers  (2026-09-06T08:49:10Z)
• Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation [68.02899606570223]
  We propose TAMP-Nav, a unified framework for efficient embodied navigation.<n>First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual coordinates.<n>Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize)
  arXiv  Detail & Related papers  (2026-08-18T08:37:08Z)
• DRIVE-Nav: Directional Reasoning, Inspection, and Verification for Efficient Open-Vocabulary Navigation [3.249609824556708]
  DRIVE-Nav is a structured framework that organizes exploration around persistent directions.<n>It reduces redundant revisits and improves path efficiency.<n>Experiments on HM3D-OVON, HM3Dv2, and MP3D demonstrate strong overall performance and consistent efficiency gains.
  arXiv  Detail & Related papers  (2026-03-30T17:12:17Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• DAgger Diffusion Navigation: DAgger Boosted Diffusion Policy for Vision-Language Navigation [73.80968452950854]
  Vision-Language Navigation in Continuous Environments (VLN-CE) requires agents to follow natural language instructions through free-form 3D spaces.<n>Existing VLN-CE approaches typically use a two-stage waypoint planning framework.<n>We propose DAgger Diffusion Navigation (DifNav) as an end-to-end optimized VLN-CE policy.
  arXiv  Detail & Related papers  (2025-08-13T02:51:43Z)
• SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigation [12.152477445938759]
  Vision-and-Language Navigation (VLN) in continuous environments requires agents to interpret natural language instructions while navigating unconstrained 3D spaces.<n>Existing VLN-CE frameworks rely on a two-stage approach: a waypoint predictor to generate waypoints and a navigator to execute movements.<n>We propose a zero-shot VLN-CE framework integrating an enhanced waypoint predictor with a Multi-modal Large Language Model (MLLM)-based navigator.
  arXiv  Detail & Related papers  (2025-03-13T05:32:57Z)
• NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning [97.88246428240872]
  Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions.<n>Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability.<n>This paper introduces a novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision.
  arXiv  Detail & Related papers  (2024-03-12T07:27:02Z)
• LangNav: Language as a Perceptual Representation for Navigation [63.90602960822604]
  We explore the use of language as a perceptual representation for vision-and-language navigation (VLN) Our approach uses off-the-shelf vision systems for image captioning and object detection to convert an agent's egocentric panoramic view at each time step into natural language descriptions.
  arXiv  Detail & Related papers  (2023-10-11T20:52:30Z)
• ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments [56.194988818341976]
  Vision-language navigation is a task that requires an agent to follow instructions to navigate in environments. We propose ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments. ETPNav yields more than 10% and 20% improvements over prior state-of-the-art on R2R-CE and RxR-CE datasets.
  arXiv  Detail & Related papers  (2023-04-06T13:07:17Z)
• Reinforced Structured State-Evolution for Vision-Language Navigation [42.46176089721314]
  Vision-and-language Navigation (VLN) task requires an embodied agent to navigate to a remote location following a natural language instruction. Previous methods usually adopt a sequence model (e.g., Transformer and LSTM) as the navigator. We propose a novel Structured state-Evolution (SEvol) model to effectively maintain the environment layout clues for VLN.
  arXiv  Detail & Related papers  (2022-04-20T07:51:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.