Nipping the Drift in the Bud: Retrospective Rectification for Robust Vision-Language Navigation

• URL: http://arxiv.org/abs/2602.06356v1
• Date: Fri, 06 Feb 2026 03:36:27 GMT
• Title: Nipping the Drift in the Bud: Retrospective Rectification for Robust Vision-Language Navigation
• Abstract summary: BudVLN is an online framework that learns from on-policy rollouts by constructing supervision to match the current state distribution.<n>BudVLN consistently mitigates distribution shift and achieves state-of-the-art performance in both Success Rate and SPL.
• Score: 26.497706746023407
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language Navigation (VLN) requires embodied agents to interpret natural language instructions and navigate through complex continuous 3D environments. However, the dominant imitation learning paradigm suffers from exposure bias, where minor deviations during inference lead to compounding errors. While DAgger-style approaches attempt to mitigate this by correcting error states, we identify a critical limitation: Instruction-State Misalignment. Forcing an agent to learn recovery actions from off-track states often creates supervision signals that semantically conflict with the original instruction. In response to these challenges, we introduce BudVLN, an online framework that learns from on-policy rollouts by constructing supervision to match the current state distribution. BudVLN performs retrospective rectification via counterfactual re-anchoring and decision-conditioned supervision synthesis, using a geodesic oracle to synthesize corrective trajectories that originate from valid historical states, ensuring semantic consistency. Experiments on the standard R2R-CE and RxR-CE benchmarks demonstrate that BudVLN consistently mitigates distribution shift and achieves state-of-the-art performance in both Success Rate and SPL.

Related papers

• Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving [37.533498955258544]
  We propose a single-pass guidance mechanism that replaces instance-level residuals with class-level latent shifts.<n>We demonstrate that LCS reduces inference latency by approximately 50% while achieving stronger command adherence.
  arXiv  Detail & Related papers  (2026-07-31T19:32:28Z)
• Rethinking State Tracking in Recurrent Models Through Error Control Dynamics [26.942965240880515]
  We show that affine recurrent networks cannot correct errors along state-separating subspaces once they preserve state representations.<n>We prove that robust state tracking is determined not only by an architecture's theoretical expressivity but crucially by its error control.
  arXiv  Detail & Related papers  (2026-05-08T13:59:15Z)
• DecoVLN: Decoupling Observation, Reasoning, and Correction for Vision-and-Language Navigation [49.233063630722334]
  We propose DecoVLN, a framework for robust streaming perception and closed-loop control in long-horizon navigation.<n>We introduce adaptive refinement mechanism that selects frames from a historical candidate pool by iteratively optimizing a unified scoring function.<n>We also introduce a state-action pair-level corrective finetuning strategy to alleviate compounding errors.
  arXiv  Detail & Related papers  (2026-03-13T16:24:37Z)
• Let's Reward Step-by-Step: Step-Aware Contrastive Alignment for Vision-Language Navigation in Continuous Environments [38.97818584066075]
  Vision-Language Navigation in Continuous Environments (VLN-CE) requires agents to learn complex reasoning from long-horizon human interactions.<n>Current training paradigms struggle to balance generalization capability, error recovery and training stability.<n>We introduce Step-Aware Contrastive Alignment (SACA), a framework designed to extract dense supervision from imperfect trajectories.
  arXiv  Detail & Related papers  (2026-03-10T14:45:50Z)
• Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration [24.562540060971273]
  Vision-Language-Action (VLA) models enable robots to perform manipulation tasks directly from natural language instructions.<n>We reveal a critical failure mode in which VLA policies continue executing visually plausible actions even when the language instruction contradicts the scene.<n>We propose Instruction-Guided Attention Recalibration (IGAR), a train-free inference-time mechanism that rebalances attention to restore the influence of language instructions.
  arXiv  Detail & Related papers  (2026-03-06T08:01:36Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)
• Vision-and-Language Navigation via Causal Learning [13.221880074458227]
  Cross-modal causal transformer (GOAT) is a pioneering solution rooted in the paradigm of causal inference. BACL and FACL modules promote unbiased learning by comprehensively mitigating potential spurious correlations. To capture global confounder features, we propose a cross-modal feature pooling module supervised by contrastive learning.
  arXiv  Detail & Related papers  (2024-04-16T02:40:35Z)
• Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning [125.61772424068903]
  Vision-and-language navigation (VLN) asks an agent to follow a given language instruction to navigate through a real 3D environment. We present a model-agnostic training paradigm, called Progressive Perturbation-aware Contrastive Learning (PROPER) to enhance the generalization ability of existing VLN agents.
  arXiv  Detail & Related papers  (2024-03-09T02:34:13Z)
• Anticipating the Unseen Discrepancy for Vision and Language Navigation [63.399180481818405]
  Vision-Language Navigation requires the agent to follow natural language instructions to reach a specific target. The large discrepancy between seen and unseen environments makes it challenging for the agent to generalize well. We propose Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency.
  arXiv  Detail & Related papers  (2022-09-10T19:04:40Z)
• Contrastive Instruction-Trajectory Learning for Vision-Language Navigation [66.16980504844233]
  A vision-language navigation (VLN) task requires an agent to reach a target with the guidance of natural language instruction. Previous works fail to discriminate the similarities and discrepancies across instruction-trajectory pairs and ignore the temporal continuity of sub-instructions. We propose a Contrastive Instruction-Trajectory Learning framework that explores invariance across similar data samples and variance across different ones to learn distinctive representations for robust navigation.
  arXiv  Detail & Related papers  (2021-12-08T06:32:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.