LiMoDE: Rethinking Lifelong Robot Manipulation from a Mixture-of-Dynamic-Experts Perspective

• URL: http://arxiv.org/abs/2606.26183v1
• Date: Wed, 24 Jun 2026 13:18:43 GMT
• Title: LiMoDE: Rethinking Lifelong Robot Manipulation from a Mixture-of-Dynamic-Experts Perspective
• Abstract summary: This paper presents an architectural perspective on the Lifelong Mixture of Dynamic Experts (textitLiMoDE)<n>Specifically, a dynamic MoE structure is first proposed in the multi-task pre-training stage to learn prior knowledge.<n>In the task adaptation stage, we design a lifelong MoE adaptation mechanism % (LiMoEAM) that learns lifelong experts and dynamically combines them with frozen ones for new tasks.
• Score: 7.2941149985927645
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building a generalist robot that can leverage prior knowledge for continuous task adaptation remains a significant challenge. Previous works alleviate the catastrophic forgetting problem by parameter-efficient fine-tuning for single-task adaptation. However, they fail to extract reusable skills and model the interaction with other skills effectively. Recent works try to address these issues by learning prompts. Differently, this paper presents an architectural perspective on the Lifelong Mixture of Dynamic Experts (\textit{LiMoDE}), a novel two-stage learning scheme for lifelong robot manipulation. Specifically, a dynamic MoE structure is first proposed in the multi-task pre-training stage to learn prior knowledge, where a varied number of heterogeneous experts are activated based on the motion information to address different short-term manipulations. Subsequently, in the task adaptation stage, we design a lifelong MoE adaptation mechanism % (LiMoEAM) that learns lifelong experts and dynamically combines them with frozen ones for new tasks, facilitating the knowledge transfer during adaptation. The proposed \textit{LiMoDE} is evaluated on both the simulated lifelong learning benchmark and real-world tasks. Extensive experiments demonstrate its effectiveness in achieving superior performance and strong lifelong adaptation by introducing a moderate number of additional trainable parameters and inference overhead.

Related papers

• LifeMem: Enabling Lifelong Experience Reuse for LLM Agents [79.11224209641739]
  LifeMem is a lifelong learning framework that enables agents to transfer knowledge across multiple environments.<n>LifeMem accumulated interaction trajectories based on underlying trajectories to extract reusable skills.<n>Results show that LifeMem enables effective experience reuse in lifelong learning, achieving both reduced forgetting on learned tasks and superior cross-task transfer.
  arXiv  Detail & Related papers  (2026-09-11T10:00:43Z)
• Spatially Generalizable Mobile Manipulation via Adaptive Experience Selection and Dynamic Imagination [23.225672159796815]
  Mobile Manipulation involves long-horizon decision-making over multi-stage compositions of heterogeneous skills.<n>Existing MM methods face two key limitations: (i) low sample efficiency, due to ineffective use of redundant data generated during long-term interactions.<n>In this paper, we address these challenges through Adaptive Experience Selection (AES) and model-based dynamic imagination.
  arXiv  Detail & Related papers  (2026-01-21T04:43:49Z)
• Mixtures of SubExperts for Large Language Continual Learning [6.425296129700846]
  Adapting Large Language Models to a continuous stream of tasks is a critical yet challenging endeavor.<n>Reusing a single set of PEFT parameters for new tasks often leads to catastrophic forgetting of prior knowledge.<n>We propose a novel adaptive PEFT method referred to as textitMixtures of SubExperts (MoSEs), a novel continual learning framework designed for minimal forgetting and efficient scalability.
  arXiv  Detail & Related papers  (2025-11-09T05:44:45Z)
• Dynamic Mixture of Progressive Parameter-Efficient Expert Library for Lifelong Robot Learning [64.88132446226474]
  A generalist agent must continuously learn and adapt throughout its lifetime, achieving efficient forward transfer while minimizing forgetting.<n>Previous work has explored parameter-efficient fine-tuning for single-task adaptation, effectively steering a frozen pretrained model with a small number of parameters.<n>We propose Dynamic Mixture of Progressive Efficient Expert Library (DMPEL) for lifelong robot learning.
  arXiv  Detail & Related papers  (2025-06-06T11:13:04Z)
• SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation [3.1997825444285457]
  Real-world robot manipulation in dynamic unstructured environments requires lifelong adaptability to evolving objects, scenes and tasks.<n>Traditional imitation learning relies on static training paradigms, which are ill-suited for lifelong adaptation.<n>We propose Skill Prompts-based HiErarchical Continual Imitation Learning (SPECI), a novel end-to-end hierarchical CIL policy architecture for robot manipulation.
  arXiv  Detail & Related papers  (2025-04-22T03:30:38Z)
• LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models [21.888139819188105]
  LLaVA-CMoE is a continual learning framework for large language models.<n> Probe-Guided Knowledge Extension mechanism determines when and where new experts should be added.<n>Probabilistic Task Locator assigns each task a dedicated, lightweight router.
  arXiv  Detail & Related papers  (2025-03-27T07:36:11Z)
• Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learning [59.001091197106085]
  Multi-Task Learning (MTL) for Vision Transformer aims at enhancing the model capability by tackling multiple tasks simultaneously.<n>Most recent works have predominantly focused on designing Mixture-of-Experts (MoE) structures and in tegrating Low-Rank Adaptation (LoRA) to efficiently perform multi-task learning.<n>We propose a novel approach dubbed Efficient Multi-Task Learning (EMTAL) by transforming a pre-trained Vision Transformer into an efficient multi-task learner.
  arXiv  Detail & Related papers  (2025-01-12T17:41:23Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning [64.55001982176226]
  LIBERO is a novel benchmark of lifelong learning for robot manipulation. We focus on how to efficiently transfer declarative knowledge, procedural knowledge, or the mixture of both. We develop an extendible procedural generation pipeline that can in principle generate infinitely many tasks.
  arXiv  Detail & Related papers  (2023-06-05T23:32:26Z)
• Hierarchical Few-Shot Imitation with Skill Transition Models [66.81252581083199]
  Few-shot Imitation with Skill Transition Models (FIST) is an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments.
  arXiv  Detail & Related papers  (2021-07-19T15:56:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.