From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation

• URL: http://arxiv.org/abs/2607.03792v1
• Date: Sat, 04 Jul 2026 09:44:57 GMT
• Title: From Region Arrival to Instance-Level Grounding in Vision-and-Language Navigation
• Abstract summary: We formalize the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding.<n>To mitigate this gap, we propose REALM, a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation.
• Score: 17.248079131598058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-and-Language Navigation (VLN) agents may satisfy conventional success criteria while still failing to establish reliable object-level grounding, because current evaluation protocols mainly reward stopping within a 3-meter radius and largely ignore the agent's final orientation and target visibility. We formalize this limitation as the Last-3-Meter Grounding Gap and introduce three instance-centric metrics to quantify proximity precision, target visibility, and final-view grounding. To mitigate this gap, we propose REALM (Region-to-Entity Alignment for Last-3-Meter Navigation), a plug-and-play, architecture-agnostic refinement module that decouples fine-grained target approaching from long-horizon navigation. REALM uses a visibility-aware stopping strategy to reduce premature termination and improve final viewpoint alignment. We further construct REVERIE-AIM, which provides object-instance-level goals and 180K short-horizon training samples for final-stage target approaching. Extensive evaluations across four diverse VLN backbones show that REALM consistently improves proximity precision and visual grounding success, demonstrating its broad applicability.

Related papers

• Deliberate Before You Fly: Vision-Guided Spatial Deliberation for UAV See-and-Reach Navigation [23.72068584471195]
  See-and-reach navigation requires an aerial agent to approach a language-specified target visible in its initial view and stop reliably near it.<n>Existing methods typically map vision-language representations directly to action outputs without explicitly modeling intermediate fine-grained spatial decisions.<n>We propose DBFly, a vision-language waypoint prediction framework that introduces explicit vision-guided spatial deliberation before waypoint generation.
  arXiv  Detail & Related papers  (2026-08-05T13:27:07Z)
• See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View [22.860978565302233]
  UAV Vision-Language Navigation (UAV-VLN) is typically formulated as a holistic search-and-reach problem.<n>We introduce UAV-VLN-FOV, a target-visible navigation task that isolates the see-and-reach stage.<n>We propose 3DG-VLN, a vision-language waypoint prediction framework guided by dynamic 3D direction cues.
  arXiv  Detail & Related papers  (2026-06-18T10:21:42Z)
• RELO: Reinforcement Learning to Localize for Visual Object Tracking [70.27265738642956]
  We introduce RELO, a REinforcement-learning-to-LOcalize method for visual object tracking.<n>RELO replaces handcrafted spatial priors with a localization policy learned over spatial positions.<n>We show that RELO achieves 57.5% AUC on LaSOText without template updates.
  arXiv  Detail & Related papers  (2026-05-08T07:34:29Z)
• Multi-stage Planning for Multi-target Surveillance using Aircrafts Equipped with Synthetic Aperture Radars Aware of Target Visibility [40.94744872735617]
  This paper presents a novel system for generating trajectories for synthetic aperture radar (SAR)-equipped aircraft.<n>A novel neural network trained with deep reinforcement learning is used to predict straight-flight segments maximizing target visibility according to the 3D terrain.<n>The system ensures high-quality multi-target SAR image acquisition aware of 3D terrain and target visibility, and real-time performance.
  arXiv  Detail & Related papers  (2026-04-18T11:04:25Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• IoUCert: Robustness Verification for Anchor-based Object Detectors [58.35703549470485]
  We introduce IoUCert, a novel formal verification framework designed specifically to overcome these bottlenecks in anchor-based object detection architectures.<n>We show that our method enables the robustness verification of realistic, anchor-based models including SSD, YOLOv2, and YOLOv3 variants against various input perturbations.
  arXiv  Detail & Related papers  (2026-03-03T14:36:46Z)
• 3DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting [12.057873540714098]
  3DGSNav is a novel framework that embeds 3D Gaussian Splatting (3DGS) as persistent memory for vision-language models (VLMs) to enhance spatial reasoning.<n>3DGSNav incrementally constructs a 3DGS representation of the environment, enabling trajectory-guided free-viewpoint rendering of frontier-aware first-person views.<n>During navigation, a real-time object detector filters potential targets, while VLM-driven active viewpoint switching performs target re-verification.
  arXiv  Detail & Related papers  (2026-02-12T16:41:26Z)
• TransBridge: Boost 3D Object Detection by Scene-Level Completion with Transformer Decoder [66.22997415145467]
  This paper presents a joint completion and detection framework that improves the detection feature in sparse areas.<n> Specifically, we propose TransBridge, a novel transformer-based up-sampling block that fuses the features from the detection and completion networks.<n>The results show that our framework consistently improves end-to-end 3D object detection, with the mean average precision (mAP) ranging from 0.7 to 1.5 across multiple methods.
  arXiv  Detail & Related papers  (2025-12-12T00:08:03Z)
• Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation [54.04601077224252]
  Embodied scene understanding requires not only comprehending visual-spatial information but also determining where to explore next in the 3D physical world.<n>underlinetextbf3D vision-language learning enables embodied agents to effectively explore and understand their environment.<n>model's versatility enables navigation using diverse input modalities, including categories, language descriptions, and reference images.
  arXiv  Detail & Related papers  (2025-07-05T14:15:52Z)
• SemNav: A Model-Based Planner for Zero-Shot Object Goal Navigation Using Vision-Foundation Models [10.671262416557704]
  Vision Foundation Models (VFMs) offer powerful capabilities for visual understanding and reasoning.<n>We present a zero-shot object goal navigation framework that integrates the perceptual strength of VFMs with a model-based planner.<n>We evaluate our approach on the HM3D dataset using the Habitat simulator and demonstrate that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-06-04T03:04:54Z)
• Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation [64.84996994779443]
  We propose a novel Affordances-Oriented Planner for continuous vision-language navigation (VLN) task. Our AO-Planner integrates various foundation models to achieve affordances-oriented low-level motion planning and high-level decision-making. Experiments on the challenging R2R-CE and RxR-CE datasets show that AO-Planner achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-07-08T12:52:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.