Joint On-and-Off Policy Learning for Vision-and-Language Navigation

• URL: http://arxiv.org/abs/2607.13461v1
• Date: Wed, 15 Jul 2026 05:46:34 GMT
• Title: Joint On-and-Off Policy Learning for Vision-and-Language Navigation
• Abstract summary: Vision-and-Language Navigation (VLN) requires an embodied agent to navigate in the physical world by adhering to natural language instructions.<n>Recent advancements in Vision-Language Models (VLM) have propelled the development of VLM-based VLN methods.<n>This paper introduces JOP-VLN, a novel VLN framework that combines off-policy imitation learning and on-policy exploration within a three-stage training pipeline.
• Score: 44.53787822402111
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-and-Language Navigation (VLN) necessitates an embodied agent to navigate in the physical world by adhering to natural language instructions. Recent advancements in Vision-Language Models (VLM) have propelled the development of VLM-based VLN methods with two predominant paradigms: (1) imitation learning (IL) on expert demonstrations, followed by the Dataset Aggregation (DAgger) algorithm to bolster error recovery capabilities; (2) reinforcement learning (RL) driven by verifiable rewards to enhance reasoning and exploration. A notable gap is the absence of integration between these two distinct paradigms. This paper introduces JOP-VLN, a novel VLN framework that synergistically combines off-policy imitation learning and on-policy exploration within a three-stage training pipeline. Initially, IL is employed on expert demonstrations to acquire basic navigation skills. Subsequently, the DAgger algorithm is utilized to generate heuristic exploration trajectories, which are then used for imitation learning to improve error recovery capabilities. Finally, a joint on-and-off policy learning framework is implemented, featuring high-entropy trajectory sampling to enhance RL training efficiency and an error-correction-prioritized trajectory sorting strategy for effective error correction. Extensive experiments demonstrate the efficacy of JOP-VLN, achieving success rates of 69.9% and 68.0% on the VLN-CE R2R and RxR benchmarks, respectively, setting a new state-of-the-art on R2R. Project page: https://qingrongh.github.io/JOP-VLN.

Related papers

• Bootstrapping Exploration with Group-Level Natural Language Feedback in Reinforcement Learning [56.29188272643489]
  We propose GOLF, an RL framework that exploits group-level language feedback to guide targeted exploration.<n>GOLF aggregates external critiques that pinpoint errors or propose targeted fixes, and intra-group attempts that supply alternative partial ideas and diverse failure patterns.<n>Experiments on both verifiable and non-verifiable benchmarks show that GOLF achieves superior performance and exploration efficiency.
  arXiv  Detail & Related papers  (2026-03-04T20:53:17Z)
• Learning to Retrieve Navigable Candidates for Efficient Vision-and-Language Navigation [15.242490558864626]
  Vision-and-Language Navigation (VLN) requires an agent to follow natural-language instructions and navigate through previously unseen environments.<n>We propose a retrieval-augmented framework to improve VLN without modifying or fine-tuning the underlying language model.
  arXiv  Detail & Related papers  (2026-02-17T17:00:11Z)
• ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigation [57.399685080574756]
  Existing MLLM-based VLN methods rely on imitation learning (IL) and often use DAgger for post-training.<n>We propose ActiveVLN, a VLN framework that explicitly enables active exploration through multi-turn RL.<n>Experiments show that ActiveVLN achieves the largest performance gains over IL baselines compared to both DAgger-based and prior RL-based post-training methods.
  arXiv  Detail & Related papers  (2025-09-16T03:31:46Z)
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning [77.34267241692706]
  Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions.<n>We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions.
  arXiv  Detail & Related papers  (2025-06-20T17:59:59Z)
• EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning [145.32076310071434]
  We propose EvolveNav, a novel embodied reasoning paradigm that realizes adaptable and generalizable navigational reasoning.<n>EvolveNav involves a two-stage training process: (1) Formalized CoT Supervised Fine-Tuning, where we train the model with curated formalized CoT labels to first activate the model's navigational reasoning capabilities, and simultaneously increase the reasoning speed; (2) Self-Reflective Post-Training, where the model is iteratively trained with its own reasoning outputs as self-enriched CoT labels to enhance the supervision diversity.
  arXiv  Detail & Related papers  (2025-06-02T11:28:32Z)
• NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning [97.88246428240872]
  Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions.<n>Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability.<n>This paper introduces a novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision.
  arXiv  Detail & Related papers  (2024-03-12T07:27:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.