Lifelong Language-Conditioned Robotic Manipulation Learning

• URL: http://arxiv.org/abs/2603.05160v1
• Date: Thu, 05 Mar 2026 13:30:33 GMT
• Title: Lifelong Language-Conditioned Robotic Manipulation Learning
• Authors: Xudong Wang, Zebin Han, Zhiyu Liu, Gan Li, Jiahua Dong, Baichen Liu, Lianqing Liu, Zhi Han,
• Abstract summary: Traditional language-conditioned manipulation agent sequential adaptation to new manipulation skills leads to catastrophic forgetting of old skills.<n>We propose SkillsCrafter, a novel robotic manipulation framework designed to continually learn multiple skills.
• Score: 19.640584848758625
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional language-conditioned manipulation agent sequential adaptation to new manipulation skills leads to catastrophic forgetting of old skills, limiting dynamic scene practical deployment. In this paper, we propose SkillsCrafter, a novel robotic manipulation framework designed to continually learn multiple skills while reducing catastrophic forgetting of old skills. Specifically, we propose a Manipulation Skills Adaptation to retain the old skills knowledge while inheriting the shared knowledge between new and old skills to facilitate learning of new skills. Meanwhile, we perform the singular value decomposition on the diverse skill instructions to obtain common skill semantic subspace projection matrices, thereby recording the essential semantic space of skills. To achieve forget-less and generalization manipulation, we propose a Skills Specialization Aggregation to compute inter-skills similarity in skill semantic subspaces, achieving aggregation of the previously learned skill knowledge for any new or unknown skill. Extensive experiments demonstrate the effectiveness and superiority of our proposed SkillsCrafter.

Related papers

• Pretrained Bayesian Non-parametric Knowledge Prior in Robotic Long-Horizon Reinforcement Learning [10.598207472087578]
  Reinforcement learning (RL) methods typically learn new tasks from scratch, often disregarding prior knowledge that could accelerate the learning process.<n>This work introduces a method that models potential primitive skill motions as having non-parametric properties with an unknown number of underlying features.<n>We utilize a non-parametric model, specifically Dirichlet Process Mixtures, enhanced with birth and merge, to pre-train a skill prior that effectively captures the diverse nature of skills.
  arXiv  Detail & Related papers  (2025-03-27T20:43:36Z)
• MaestroMotif: Skill Design from Artificial Intelligence Feedback [67.17724089381056]
  MaestroMotif is a method for AI-assisted skill design, which yields high-performing and adaptable agents.<n>We present MaestroMotif, a method for AI-assisted skill design, which yields high-performing and adaptable agents.
  arXiv  Detail & Related papers  (2024-12-11T16:59:31Z)
• SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions [48.003320766433966]
  This work introduces Skill Discovery from Local Dependencies (Skild) Skild develops a novel skill learning objective that explicitly encourages the mastering of skills that induce different interactions within an environment. We evaluate Skild in several domains with challenging, long-horizon sparse reward tasks including a realistic simulated household robot domain.
  arXiv  Detail & Related papers  (2024-10-24T04:01:59Z)
• Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning [39.991887534269445]
  Disentangled Unsupervised Skill Discovery (DUSDi) is a method for learning disentangled skills that can be efficiently reused to solve downstream tasks. DUSDi decomposes skills into disentangled components, where each skill component only affects one factor of the state space. DUSDi successfully learns disentangled skills, and significantly outperforms previous skill discovery methods when it comes to applying the learned skills to solve downstream tasks.
  arXiv  Detail & Related papers  (2024-10-15T04:13:20Z)
• Language Guided Skill Discovery [56.84356022198222]
  We introduce Language Guided Skill Discovery (LGSD) to maximize semantic diversity between skills.<n>LGSD takes user prompts as input and outputs a set of semantically distinctive skills.<n>We demonstrate that LGSD enables legged robots to visit different user-intended areas on a plane by simply changing the prompt.
  arXiv  Detail & Related papers  (2024-06-07T04:25:38Z)
• C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters [49.83342243500835]
  We present C$cdot$ASE, an efficient framework that learns conditional Adversarial Skill Embeddings for physics-based characters. C$cdot$ASE divides the heterogeneous skill motions into distinct subsets containing homogeneous samples for training a low-level conditional model. The skill-conditioned imitation learning naturally offers explicit control over the character's skills after training.
  arXiv  Detail & Related papers  (2023-09-20T14:34:45Z)
• Choreographer: Learning and Adapting Skills in Imagination [60.09911483010824]
  We present Choreographer, a model-based agent that exploits its world model to learn and adapt skills in imagination. Our method decouples the exploration and skill learning processes, being able to discover skills in the latent state space of the model. Choreographer is able to learn skills both from offline data, and by collecting data simultaneously with an exploration policy.
  arXiv  Detail & Related papers  (2022-11-23T23:31:14Z)
• One After Another: Learning Incremental Skills for a Changing World [19.051800747558794]
  We propose a new framework for skill discovery, where skills are learned one after another in an incremental fashion. We demonstrate experimentally that in both evolving and static environments, incremental skills significantly outperform current state-of-the-art skill discovery methods.
  arXiv  Detail & Related papers  (2022-03-21T17:55:21Z)
• A Simple Approach to Continual Learning by Transferring Skill Parameters [25.705923249267055]
  We show how to continually acquire robotic manipulation skills without forgetting, and using far fewer samples than needed to train them from scratch. Given an appropriate curriculum, we show how to continually acquire robotic manipulation skills without forgetting, and using far fewer samples than needed to train them from scratch.
  arXiv  Detail & Related papers  (2021-10-19T20:44:20Z)
• Discovering Generalizable Skills via Automated Generation of Diverse Tasks [82.16392072211337]
  We propose a method to discover generalizable skills via automated generation of a diverse set of tasks. As opposed to prior work on unsupervised discovery of skills, our method pairs each skill with a unique task produced by a trainable task generator. A task discriminator defined on the robot behaviors in the generated tasks is jointly trained to estimate the evidence lower bound of the diversity objective. The learned skills can then be composed in a hierarchical reinforcement learning algorithm to solve unseen target tasks.
  arXiv  Detail & Related papers  (2021-06-26T03:41:51Z)
• Relative Variational Intrinsic Control [11.328970848714919]
  Relative Variational Intrinsic Control (RVIC) incentivizes learning skills that are distinguishable in how they change the agent's relationship to its environment. We show how RVIC skills are more useful than skills discovered by existing methods when used in hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2020-12-14T18:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.