User-Feedback-Driven Continual Adaptation for Vision-and-Language Navigation

• URL: http://arxiv.org/abs/2512.10322v1
• Date: Thu, 11 Dec 2025 06:11:45 GMT
• Title: User-Feedback-Driven Continual Adaptation for Vision-and-Language Navigation
• Authors: Yongqiang Yu, Xuhui Li, Hazza Mahmood, Jinxing Zhou, Haodong Hong, Longtao Jiang, Zhiqiang Xu, Qi Wu, Xiaojun Chang,
• Abstract summary: Vision-and-Language Navigation (VLN) requires agents to navigate complex environments by following natural-language instructions.<n>Current GSA-VLN frameworks exclude user feedback, relying solely on unsupervised adaptation from repeated environmental exposure.<n>We introduce a user-feedback-driven adaptation framework that extends GSA-VLN by systematically integrating human interactions into continual learning.
• Score: 50.51140007672333
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision-and-Language Navigation (VLN) requires agents to navigate complex environments by following natural-language instructions. General Scene Adaptation for VLN (GSA-VLN) shifts the focus from zero-shot generalization to continual, environment-specific adaptation, narrowing the gap between static benchmarks and real-world deployment. However, current GSA-VLN frameworks exclude user feedback, relying solely on unsupervised adaptation from repeated environmental exposure. In practice, user feedback offers natural and valuable supervision that can significantly enhance adaptation quality. We introduce a user-feedback-driven adaptation framework that extends GSA-VLN by systematically integrating human interactions into continual learning. Our approach converts user feedback-navigation instructions and corrective signals-into high-quality, environment-aligned training data, enabling efficient and realistic adaptation. A memory-bank warm-start mechanism further reuses previously acquired environmental knowledge, mitigating cold-start degradation and ensuring stable redeployment. Experiments on the GSA-R2R benchmark show that our method consistently surpasses strong baselines such as GR-DUET, improving navigation success and path efficiency. The memory-bank warm start stabilizes early navigation and reduces performance drops after updates. Results under both continual and hybrid adaptation settings confirm the robustness and generality of our framework, demonstrating sustained improvement across diverse deployment conditions.

Related papers

• Adapting to Fragmented and Evolving Data: A Fisher Information Perspective [0.0]
  FADE is a lightweight framework for robust learning under dynamic environments.<n>It employs a shift-aware regularization mechanism anchored in Fisher information geometry.<n>FADE operates online with fixed memory and no access to target labels.
  arXiv  Detail & Related papers  (2025-07-25T06:50:09Z)
• NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environments [67.18144414660681]
  Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to execute sequential navigation actions in complex environments guided by natural language instructions.<n>Inspired by human cognition, we present NavMorph, a self-evolving world model framework that enhances environmental understanding and decision-making in VLN-CE tasks.
  arXiv  Detail & Related papers  (2025-06-30T02:20:00Z)
• VEM: Environment-Free Exploration for Training GUI Agent with Value Environment Model [34.98047665907545]
  We propose an environment-free RL framework that decouples value estimation from policy optimization.<n>The framework operates in two stages: (1) pretraining VEM to estimate long-term action utilities and (2) guiding policy exploration with frozen VEM signals.<n> evaluated on Android-in-the-Wild benchmarks, VEM achieves state-of-the-art performance in both offline and online settings.
  arXiv  Detail & Related papers  (2025-02-26T07:52:02Z)
• Generalized Back-Stepping Experience Replay in Sparse-Reward Environments [2.6887381380521878]
  Back-stepping experience replay (BER) is a reinforcement learning technique that can accelerate learning efficiency in reversible environments.<n>The original algorithm is designed for a dense-reward environment that does not require complex exploration.<n>We propose an enhanced version of BER called Generalized BER (GBER), which extends the original algorithm to sparse-reward environments.
  arXiv  Detail & Related papers  (2024-12-20T03:31:23Z)
• AURO: Reinforcement Learning for Adaptive User Retention Optimization in Recommender Systems [25.18963930580529]
  Reinforcement Learning (RL) has garnered increasing attention for its ability to optimize user retention in recommender systems.<n>This paper introduces a novel approach called textbfAdaptive textbfUser textbfRetention textbfOptimization (AURO) to address this challenge.
  arXiv  Detail & Related papers  (2023-10-06T02:45:21Z)
• Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust? [91.682459306359]
  In continual learning (CL), an AI agent learns from non-stationary data streams under dynamic environments. In this paper, a novel CL framework is proposed to achieve robust generalization to dynamic environments while retaining past knowledge. The generalization and memorization performance of the proposed framework are theoretically analyzed.
  arXiv  Detail & Related papers  (2023-09-18T21:00:01Z)
• RE-MOVE: An Adaptive Policy Design for Robotic Navigation Tasks in Dynamic Environments via Language-Based Feedback [56.219221064727016]
  Reinforcement learning-based policies for continuous control robotic navigation tasks often fail to adapt to changes in the environment during real-time deployment. We propose a novel approach called RE-MOVE to adapt already trained policy to real-time changes in the environment without re-training via utilizing a language-based feedback.
  arXiv  Detail & Related papers  (2023-03-14T04:20:59Z)
• Anticipating the Unseen Discrepancy for Vision and Language Navigation [63.399180481818405]
  Vision-Language Navigation requires the agent to follow natural language instructions to reach a specific target. The large discrepancy between seen and unseen environments makes it challenging for the agent to generalize well. We propose Unseen Discrepancy Anticipating Vision and Language Navigation (DAVIS) that learns to generalize to unseen environments via encouraging test-time visual consistency.
  arXiv  Detail & Related papers  (2022-09-10T19:04:40Z)
• Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration [83.96729205383501]
  We introduce prompt-based learning to achieve fast adaptation for language embeddings. Our model can adapt to diverse vision-language navigation tasks, including VLN and REVERIE.
  arXiv  Detail & Related papers  (2022-03-08T11:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.