DV-VLN: Dual Verification for Reliable LLM-Based Vision-and-Language Navigation

• URL: http://arxiv.org/abs/2601.18492v1
• Date: Mon, 26 Jan 2026 13:47:55 GMT
• Title: DV-VLN: Dual Verification for Reliable LLM-Based Vision-and-Language Navigation
• Abstract summary: Vision-and-Language Navigation (VLN) requires an embodied agent to navigate in a complex 3D environment according to natural language instructions.<n>Recent progress in large language models (LLMs) has enabled language-driven navigation with improved interpretability.<n>We propose DV-VLN, a new VLN framework that follows a generate-then-verify paradigm.
• Score: 18.493700097379186
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-and-Language Navigation (VLN) requires an embodied agent to navigate in a complex 3D environment according to natural language instructions. Recent progress in large language models (LLMs) has enabled language-driven navigation with improved interpretability. However, most LLM-based agents still rely on single-shot action decisions, where the model must choose one option from noisy, textualized multi-perspective observations. Due to local mismatches and imperfect intermediate reasoning, such decisions can easily deviate from the correct path, leading to error accumulation and reduced reliability in unseen environments. In this paper, we propose DV-VLN, a new VLN framework that follows a generate-then-verify paradigm. DV-VLN first performs parameter-efficient in-domain adaptation of an open-source LLaMA-2 backbone to produce a structured navigational chain-of-thought, and then verifies candidate actions with two complementary channels: True-False Verification (TFV) and Masked-Entity Verification (MEV). DV-VLN selects actions by aggregating verification successes across multiple samples, yielding interpretable scores for reranking. Experiments on R2R, RxR (English subset), and REVERIE show that DV-VLN consistently improves over direct prediction and sampling-only baselines, achieving competitive performance among language-only VLN agents and promising results compared with several cross-modal systems.Code is available at https://github.com/PlumJun/DV-VLN.

Related papers

• Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection [0.27998963147546135]
  We introduce a lightweight federated MIL-VLM cascade in which only a compact MIL scorer is trained across clients, while a frozen VLM verifies high-scoring suspect segments post hoc.<n>We study two VLM feedback interfaces: parsed text-generation decisions and a logit-based interface that extracts a continuous anomaly score from next-token Yes/No probabilities.<n>In contrast, the logit interface provides a fixed-free signal that improves both frame-level AUC and frame-level AP over the MIL baseline across both VLMs.
  arXiv  Detail & Related papers  (2026-09-07T13:04:00Z)
• VerNav: Verifier-First Low-Latency Vision-and-Language Navigation [7.190348268664948]
  Vision-and-Language Navigation (VLN) requires an agent to navigate through unseen 3D environments according to natural-latency instructions.<n>We propose VerNav, a verifier-first framework for low-language LLM-based VLN.
  arXiv  Detail & Related papers  (2026-09-01T08:45:15Z)
• CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigation [10.221757123615786]
  Vision-language navigation (VLN) has recently advanced with large language and multimodal models, enabling agents to follow natural-language instructions in unseen environments without training a task-specific navigation policy.<n>Most existing VLN methods rely on an open-loop decision-execution approach, where candidate actions are generated from instructions and observations but are rarely verified or corrected before execution.<n>We propose Closed-loop Self-verified Retrieval-augmented Reasoning (CLOSER), a training-policy-free method that sequentially performs action reasoning, reliability verification, targeted retrieval, and action correction in a closed-loop manner before executing concrete
  arXiv  Detail & Related papers  (2026-06-24T05:57:42Z)
• More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Models [74.10138874771852]
  We propose PeRL-VL (Perception and Reasoning Learning for Vision-Language Models), a decoupled framework that separately improves visual perception and textual reasoning on top of RLVR.<n>For perception, PeRL-VL introduces a VLM-based description reward that scores the model's self-generated image descriptions for faithfulness and sufficiency.<n>For reasoning, PeRL-VL adds a text-only Reasoning SFT stage on logic-rich chain-of-thought data, enhancing coherence and logical consistency independently of vision.
  arXiv  Detail & Related papers  (2025-12-13T23:06:18Z)
• dVLM-AD: Enhance Diffusion Vision-Language-Model for Driving via Controllable Reasoning [69.36145467833498]
  We introduce dVLM-AD, a diffusion-based vision-language model that unifies perception, structured reasoning, and low-level planning for end-to-end driving.<n> evaluated on nuScenes and WOD-E2E, dVLM-AD yields more consistent reasoning-action pairs and achieves planning performance comparable to existing driving VLM/VLA systems.
  arXiv  Detail & Related papers  (2025-12-04T05:05:41Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)
• COST: Contrastive One-Stage Transformer for Vision-Language Small Object Tracking [52.62149024881728]
  We propose a contrastive one-stage transformer fusion framework for vision-language (VL) tracking.<n>We introduce a contrastive alignment strategy that maximizes mutual information between a video and its corresponding language description.<n>By leveraging a visual-linguistic transformer, we establish an efficient multi-modal fusion and reasoning mechanism.
  arXiv  Detail & Related papers  (2025-04-02T03:12:38Z)
• MVL-SIB: A Massively Multilingual Vision-Language Benchmark for Cross-Modal Topical Matching [19.13134780850277]
  We introduce MVL-SIB, a massively multilingual vision-language benchmark that evaluates cross-modal and text-only topical matching across 205 languages.<n>Our results reveal that LVLMs struggle in cross-modal topic matching in lower-resource languages, performing no better than chance on languages like N'Koo.
  arXiv  Detail & Related papers  (2025-02-18T13:40:05Z)
• NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning [97.88246428240872]
  Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions.<n>Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability.<n>This paper introduces a novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision.
  arXiv  Detail & Related papers  (2024-03-12T07:27:02Z)
• Adapting Pre-trained Language Models to Vision-Language Tasks via Dynamic Visual Prompting [83.21164539349273]
  Pre-trained language models (PLMs) have played an increasing role in multimedia research. In this paper, we focus on exploring PLMs as a stand-alone model for vision-language reasoning tasks. We propose a novel transfer learning approach for PLMs, termed Dynamic Visual Prompting (DVP)
  arXiv  Detail & Related papers  (2023-06-01T07:19:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.