DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models

• URL: http://arxiv.org/abs/2605.07494v1
• Date: Fri, 08 May 2026 09:32:05 GMT
• Title: DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models
• Abstract summary: Continual learning enables vision-language models to accumulate knowledge and adapt to evolving tasks without retraining from scratch.<n>Most existing methods rely on fixed architectures with statically allocated parameters, which limits adaptation to new domains and aggravates catastrophic forgetting.<n>We propose DIMoE-Adapters, a Dynamic Incremental Mixture-of-Experts Adapters framework that introduces a dynamic expert evolution paradigm to balance stability and plasticity.
• Score: 63.58458696122293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual learning enables vision-language models to accumulate knowledge and adapt to evolving tasks without retraining from scratch. However, in multi-domain task-incremental learning, large domain shifts intensify the stability-plasticity dilemma. Most existing methods rely on fixed architectures with statically allocated parameters, which limits adaptation to new domains and aggravates catastrophic forgetting. To address these challenges, we propose DIMoE-Adapters, a Dynamic Incremental Mixture-of-Experts Adapters framework that introduces a dynamic expert evolution paradigm to balance stability and plasticity. This paradigm is implemented through two collaborative components: Self-Calibrated Expert Evolution (SCEE) and Prototype-Guided Expert Selection (PGES). SCEE constructs and evolves a sparse expert pool through expert optimization dynamics, improving plasticity while reducing redundant capacity. PGES controls expert utilization based on the pool shaped by SCEE, improving stability across both previously encountered and unseen tasks. Extensive experiments show that DIMoE-Adapters outperforms previous state-of-the-art methods across various settings.

Related papers

• Self-Improving Large Language Models via Progressive Experience Evolution [52.03479747358687]
  We propose textbfSPEE (textbfSelf-textbfProgressive textbfExperience textbfEvolution), a unified post-training framework that sequentially performs explicit experience evolution followed by implicit policy optimization.<n> Experiments on five mathematical reasoning benchmarks demonstrate that SPEE consistently outperforms both test-time and training-time self-evolution baselines across three model scales.
  arXiv  Detail & Related papers  (2026-08-03T12:27:32Z)
• MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution [49.24192816558635]
  Large language models (LLMs) exhibit strong reasoning capabilities, yet most LLM-based agents are unable to improve through task interactions.<n>Existing experience-driven methods often rely on memory or executors without enhancing the model's ability to learn.<n>We propose MetaEvo, a two-stage framework for continual agent evolution.
  arXiv  Detail & Related papers  (2026-05-29T09:31:39Z)
• Towards Adaptive Continual Model Merging via Manifold-Aware Expert Evolution [33.44708543696699]
  MADE-IT orchestrates expert management and activation by grounding intrinsic expert representations in manifold geometry.<n>Experiments demonstrate that MADE-IT consistently outperforms strong baselines in accuracy and robustness across long-horizon and shuffled task sequences.
  arXiv  Detail & Related papers  (2026-04-24T11:35:53Z)
• Training-Free Diffusion-Driven Modeling of Pareto Set Evolution for Dynamic Multiobjective Optimization [15.780107358426745]
  This paper proposes DD-DMOEA, a training-free diffusion-based dynamic response mechanism for DMOPs.<n>A knee-point-based auxiliary strategy is used to specify the target region in the new environment.<n>Experiments on the CEC2018 dynamic multiobjective benchmarks show that DD-DMOEA achieves competitive or better convergence-diversity performance.
  arXiv  Detail & Related papers  (2026-03-23T11:09:58Z)
• SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference [38.037874715181964]
  We propose SALAAD, a plug-and-play framework that induces sparse and low-rank structures during training.<n>Experiments across model scales show that SALAAD substantially reduces memory consumption during deployment.<n>A single training run yields a continuous spectrum of model capacities, enabling smooth and elastic deployment across diverse memory budgets.
  arXiv  Detail & Related papers  (2026-02-01T00:00:11Z)
• Test-Time Mixture of World Models for Embodied Agents in Dynamic Environments [29.514831254621438]
  Test-time Mixture of World Models (TMoW) is a framework that enhances adaptability to unseen and evolving domains.<n>TMoW updates its routing function over world models at test time, unlike conventional MoE where the function remains fixed.<n>We evaluate TMoW on VirtualHome, ALFWorld, and RLBench benchmarks, demonstrating strong performance in both zero-shot adaptation and few-shot expansion scenarios.
  arXiv  Detail & Related papers  (2026-01-30T07:06:40Z)
• Rethinking the Role of Dynamic Sparse Training for Scalable Deep Reinforcement Learning [58.533203990515034]
  Scaling neural networks has driven breakthrough advances in machine learning, yet this paradigm fails in deep reinforcement learning (DRL)<n>We show that dynamic sparse training strategies provide module-specific benefits that complement the primary scalability foundation established by architectural improvements.<n>We finally distill these insights into Module-Specific Training (MST), a practical framework that exploits the benefits of architectural improvements and demonstrates substantial scalability gains across diverse RL algorithms without algorithmic modifications.
  arXiv  Detail & Related papers  (2025-10-14T03:03:08Z)
• Dynamic Experts Search: Enhancing Reasoning in Mixture-of-Experts LLMs at Test Time [35.31371938688921]
  Test-Time Scaling (TTS) enhances the reasoning ability of large language models (LLMs) by allocating additional computation during inference.<n>We propose Dynamic Experts Search (DES), a TTS strategy that elevates expert activation into a controllable dimension of the search space.
  arXiv  Detail & Related papers  (2025-09-26T16:49:10Z)
• AIM: Adaptive Intra-Network Modulation for Balanced Multimodal Learning [55.56234913868664]
  We propose Adaptive Intra-Network Modulation (AIM) to improve balanced modality learning.<n>AIM accounts for differences in optimization state across parameters and depths within the network during modulation.<n>We show that AIM outperforms state-of-the-art imbalanced modality learning methods across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-08-27T10:53:36Z)
• Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization [72.81319836138347]
  Online continual learning (OCL) seeks to learn new tasks from data streams that appear only once, while retaining knowledge of previously learned tasks.<n>Most existing methods rely on replay, focusing on enhancing memory retention through regularization or distillation.<n>We introduce a plug-and-play module, S6MOD, which can be integrated into most existing methods and directly improve adaptability.
  arXiv  Detail & Related papers  (2024-12-24T05:25:21Z)
• Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning [115.79349923044663]
  Few-shot class-incremental learning (FSCIL) aims to incrementally learn novel classes from limited examples.<n>Existing methods face a critical dilemma: static architectures rely on a fixed parameter space to learn from data that arrive sequentially, prone to overfitting to the current session.<n>In this study, we explore the potential of Selective State Space Models (SSMs) for FSCIL.
  arXiv  Detail & Related papers  (2024-07-08T17:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.