Task-Differentiated Atomic Skill Expansion and Routing for Continual Learning Across Highly Heterogeneous Tasks

• URL: http://arxiv.org/abs/2606.21307v1
• Date: Fri, 19 Jun 2026 10:41:30 GMT
• Title: Task-Differentiated Atomic Skill Expansion and Routing for Continual Learning Across Highly Heterogeneous Tasks
• Abstract summary: Continual learning (CL) is commonly studied under the assumption that sequential tasks are semantically related or structurally similar.<n>textttTASER is a framework that determines how many new atomic skills to introduce for each task and which skills to activate.<n>Experiments on textttHeteroCLBench show that textttTASER consistently outperforms strong baselines by improving plasticity and reducing catastrophic forgetting.
• Score: 15.331986169435893
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning (CL) is commonly studied under the assumption that sequential tasks are semantically related or structurally similar. However, in highly heterogeneous settings, where tasks differ substantially in reasoning patterns and input-output formats, existing methods often suffer from catastrophic forgetting and inefficient capacity allocation. To address this challenge, we propose Task-differentiated Atomic Skill Expansion and Routing (\texttt{TASER}), a CL framework that jointly determines how many new atomic skills to introduce for each task and which skills to activate. The framework first uses atomic skill incremental learning to dynamically expand capacity based on task divergence and model uncertainty. It then applies orthogonality-enhanced skill detection to ensure these skills remain semantically distinct and independently reusable. Finally, a skill dynamic routing mechanism composes task-relevant skills through lightweight task-conditioned gating. We further introduce \texttt{HeteroCLBench}, a highly heterogeneous benchmark for CL, comprising 19 diverse tasks across 9 cognitive dimensions under a standardized sequential protocol. Experiments on \texttt{HeteroCLBench} show that \texttt{TASER} consistently outperforms strong baselines by improving plasticity and reducing catastrophic forgetting.

Related papers

• SkillMemo: Expert-guided Skill Memory Framework for Compositional Embodied Manipulation [72.14179740056073]
  Embodied visuomotor models, including Diffusion Policy (DP) and Vision-Language-Action (VLA) models, have demonstrated promising performance on robotic manipulation benchmarks.<n>We propose Skill-Based Memory (SkillMemo) framework that implicitly decomposes long-horizon demonstrations into latent atomic skills.
  arXiv  Detail & Related papers  (2026-08-06T12:46:13Z)
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control [21.97625864571452]
  Language-conditioned Imitation Learning (IL) is essential for enabling robots to perform complex tasks following natural language instructions.<n>We propose Dual-Process Atomic Skill Learning (DASL), a novel hierarchical imitation learning framework that decouples slow semantic reasoning from real-time motion control.<n>DASL significantly outperforms state-of-the-art baselines, excelling in skill acquisition and compositional generalization to unseen instructions.
  arXiv  Detail & Related papers  (2026-07-12T07:38:53Z)
• Learning New Tasks via Reusable Skills: Skill-Compositional Experts for Embodied Continual Learning [60.37693496953694]
  Embodied Continual Learning (ECL) aims to enable robots to continually acquire new manipulation tasks.<n>ECL suffers from more severe catastrophic forgetting.<n>Key challenge in ECL lies in structured skill reuse across continually evolving tasks.
  arXiv  Detail & Related papers  (2026-06-14T09:02:49Z)
• SkillComposer: Learning to Evolve Agent Skills for Specification and Generalization [99.18305770138677]
  Agent skills are reusable strategies that guide agent reasoning and action.<n>Current skill construction methods treat the problem as one-shot extraction.<n>We introduce SkillComposer, a framework that decomposes skill construction into three learnable operations.
  arXiv  Detail & Related papers  (2026-06-04T12:16:08Z)
• CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning [16.21619173438734]
  Multimodal Large Language Models (MLLMs) unify heterogeneous vision-language tasks under a shared generative framework via instruction tuning.<n>Existing methods either update all tasks with a shared parameter set or allocate dedicated modules for each new task.<n>CRAM mitigates catastrophic forgetting across tasks by isolating task-specific patterns into independent modules.
  arXiv  Detail & Related papers  (2026-06-01T17:11:30Z)
• Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulation [45.91196295385836]
  Cross-task generalization is a core challenge in open-world robotic manipulation.<n>We propose Decompose and Recompose, a skill reasoning framework using atomic skill-action pairs as intermediate representations.<n>Our approach decomposes seen demonstrations into interpretable skill--action alignments, enabling the model to recompose these skills for unseen tasks.
  arXiv  Detail & Related papers  (2026-05-02T13:55:28Z)
• CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning [80.18781219542016]
  Continual Learning (CL) empowers AI models to continuously learn from sequential task streams.<n>Recent parameter-efficient fine-tuning (PEFT)-based CL methods have garnered increasing attention due to their superior performance.<n>We propose Cross-subspace Knowledge Alignment and Aggregation (CKAA) to enhance robustness against misleading task-ids.
  arXiv  Detail & Related papers  (2025-07-13T03:11:35Z)
• SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation [3.1997825444285457]
  Real-world robot manipulation in dynamic unstructured environments requires lifelong adaptability to evolving objects, scenes and tasks.<n>Traditional imitation learning relies on static training paradigms, which are ill-suited for lifelong adaptation.<n>We propose Skill Prompts-based HiErarchical Continual Imitation Learning (SPECI), a novel end-to-end hierarchical CIL policy architecture for robot manipulation.
  arXiv  Detail & Related papers  (2025-04-22T03:30:38Z)
• SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution [75.2573501625811]
  Diffusion models have demonstrated strong potential for robotic trajectory planning. generating coherent trajectories from high-level instructions remains challenging. We propose SkillDiffuser, an end-to-end hierarchical planning framework.
  arXiv  Detail & Related papers  (2023-12-18T18:16:52Z)
• Combining Modular Skills in Multitask Learning [149.8001096811708]
  A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. In this work, we assume each task is associated with a subset of latent discrete skills from a (potentially small) inventory. We find that the modular design of a network significantly increases sample efficiency in reinforcement learning and few-shot generalisation in supervised learning.
  arXiv  Detail & Related papers  (2022-02-28T16:07:19Z)
• Adversarial Continual Learning [99.56738010842301]
  We propose a hybrid continual learning framework that learns a disjoint representation for task-invariant and task-specific features. Our model combines architecture growth to prevent forgetting of task-specific skills and an experience replay approach to preserve shared skills.
  arXiv  Detail & Related papers  (2020-03-21T02:08:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.