All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptation

• URL: http://arxiv.org/abs/2603.14276v1
• Date: Sun, 15 Mar 2026 08:18:54 GMT
• Title: All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptation
• Abstract summary: We formalize the all-day multi-scenes lifelong VLN (AML-VLN) problem.<n>We propose Tucker Adaptation (TuKA), which represents the multi-hierarchical navigation knowledge as a high-order tensor.<n>We also develop a VLN agent named AlldayWalker, which continually learns across multiple navigation scenarios.
• Score: 18.03268198431796
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deploying vision-and-language navigation (VLN) agents requires adaptation across diverse scenes and environments, but fine-tuning on a specific scenario often causes catastrophic forgetting in others, which severely limits flexible long-term deployment. We formalize this challenge as the all-day multi-scenes lifelong VLN (AML-VLN) problem. Existing parameter-efficient adapters (e.g., LoRA and its variants) are limited by their two-dimensional matrix form, which fails to capture the multi-hierarchical navigation knowledge spanning multiple scenes and environments. To address this, we propose Tucker Adaptation (TuKA), which represents the multi-hierarchical navigation knowledge as a high-order tensor and leverages Tucker decomposition to decouple the knowledge into shared subspaces and scenario-specific experts. We further introduce a decoupled knowledge incremental learning strategy to consolidate shared subspaces while constraining specific experts for decoupled lifelong learning. Building on TuKA, we also develop a VLN agent named AlldayWalker, which continually learns across multiple navigation scenarios, achieving all-day multi-scenes navigation. Extensive experiments show that AlldayWalker consistently outperforms state-of-the-art baselines.

Related papers

• AllDayNav: Lifelong Navigation via Real-World Reinforcement Learning [15.951251568937074]
  We propose AllDayNav, a lifelong self-learning navigation framework that implicitly encodes scene dynamics into the billion-scale parameters of a large model via reinforcement learning.<n>Experiments show that AllDayNav achieves success rates approaching $100%$ and consistently surpasses strong map-based, VLM, and RL baselines in path efficiency and robustness.
  arXiv  Detail & Related papers  (2026-06-09T14:35:53Z)
• AgentVLN: Towards Agentic Vision-and-Language Navigation [78.739525400071]
  Vision-and-Language Navigation (VLN) requires an embodied agent to ground complex natural-language instructions into long-horizon navigation in unseen environments.<n>In this paper, we propose AgentVLN, a novel and efficient embodied navigation framework that can be deployed on edge computing platforms.
  arXiv  Detail & Related papers  (2026-03-18T12:43:47Z)
• Lifelong Embodied Navigation Learning [22.062368593367623]
  Embodied navigation agents powered by large language models have shown strong performance on individual tasks but struggle to continually acquire new navigation skills.<n>We formalize this challenge as lifelong embodied navigation learning (LENL), where an agent is required to adapt to a sequence of navigation tasks.<n>We propose Uni-Walker, a lifelong embodied navigation framework that decouples navigation knowledge into task-shared and task-specific components.
  arXiv  Detail & Related papers  (2026-03-06T09:25:26Z)
• Towards Multimodal Lifelong Understanding: A Dataset and Agentic Baseline [58.585692088008905]
  MM-Lifelong is a dataset designed for Multimodal Lifelong Understanding.<n>Comprising 181.1 hours of footage, it is structured across Day, Week, and Month scales to capture varying temporal densities.
  arXiv  Detail & Related papers  (2026-03-05T18:52:12Z)
• Enhancing Vision-Language Navigation with Multimodal Event Knowledge from Real-World Indoor Tour Videos [15.251897505310682]
  We propose an event-centric knowledge enhancement strategy for automated process knowledge mining and feature fusion.<n>We extract unstructured video streams into structured semantic-action-effect events to serve as explicit episodic memory.<n> Experiments on REVERIE, R2R, and R2R-CE benchmarks demonstrate the efficiency of our strategy.
  arXiv  Detail & Related papers  (2026-02-27T11:38:06Z)
• VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory [43.2995099083993]
  VLA models have shown promising potential in embodied navigation by unifying perception and planning.<n>Most existing VLA models rely on reactive mappings directly from observations to actions.<n>We propose VLingNav, a VLA model for embodied navigation grounded in linguistic-driven cognition.
  arXiv  Detail & Related papers  (2026-01-13T15:43:43Z)
• Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents [43.5771856761934]
  Vision-and-Language Navigation (VLN) poses significant challenges for agents to interpret natural language instructions and navigate complex 3D environments.<n>We propose SkillNav, a modular framework that introduces structured, skill-based reasoning into Transformer-based VLN agents.
  arXiv  Detail & Related papers  (2025-08-11T05:50:30Z)
• DISCO: Embodied Navigation and Interaction via Differentiable Scene Semantics and Dual-level Control [53.80518003412016]
  Building a general-purpose intelligent home-assistant agent skilled in diverse tasks by human commands is a long-term blueprint of embodied AI research. We study primitive mobile manipulations for embodied agents, i.e. how to navigate and interact based on an instructed verb-noun pair. We propose DISCO, which features non-trivial advancements in contextualized scene modeling and efficient controls.
  arXiv  Detail & Related papers  (2024-07-20T05:39:28Z)
• OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation [96.46961207887722]
  OVER-NAV aims to go over and beyond the current arts of IVLN techniques. To fully exploit the interpreted navigation data, we introduce a structured representation, coded Omnigraph.
  arXiv  Detail & Related papers  (2024-03-26T02:34:48Z)
• ESceme: Vision-and-Language Navigation with Episodic Scene Memory [72.69189330588539]
  Vision-and-language navigation (VLN) simulates a visual agent that follows natural-language navigation instructions in real-world scenes. We introduce a mechanism of Episodic Scene memory (ESceme) for VLN that wakes an agent's memories of past visits when it enters the current scene.
  arXiv  Detail & Related papers  (2023-03-02T07:42:07Z)
• Towards Versatile Embodied Navigation [120.73460380993305]
  Vienna is a versatile embodied navigation agent that simultaneously learns to perform the four navigation tasks with one model. We empirically demonstrate that, compared with learning each visual navigation task individually, our agent achieves comparable or even better performance with reduced complexity.
  arXiv  Detail & Related papers  (2022-10-30T11:53:49Z)
• History Aware Multimodal Transformer for Vision-and-Language Navigation [96.80655332881432]
  Vision-and-language navigation (VLN) aims to build autonomous visual agents that follow instructions and navigate in real scenes. We introduce a History Aware Multimodal Transformer (HAMT) to incorporate a long-horizon history into multimodal decision making.
  arXiv  Detail & Related papers  (2021-10-25T22:54:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.