SEE: Continual Fine-tuning with Sequential Ensemble of Experts

• URL: http://arxiv.org/abs/2504.06664v1
• Date: Wed, 09 Apr 2025 07:56:56 GMT
• Title: SEE: Continual Fine-tuning with Sequential Ensemble of Experts
• Authors: Zhilin Wang, Yafu Li, Xiaoye Qu, Yu Cheng,
• Abstract summary: Continual fine-tuning of large language models (LLMs) suffers from catastrophic forgetting.<n>We introduce the Sequential Ensemble of Experts (SEE) framework.<n>SEE removes the need for an additional router, allowing each expert to independently decide whether a query should be handled.
• Score: 25.96255683276355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual fine-tuning of large language models (LLMs) suffers from catastrophic forgetting. Rehearsal-based methods mitigate this problem by retaining a small set of old data. Nevertheless, they still suffer inevitable performance loss. Although training separate experts for each task can help prevent forgetting, effectively assembling them remains a challenge. Some approaches use routers to assign tasks to experts, but in continual learning, they often require retraining for optimal performance. To address these challenges, we introduce the Sequential Ensemble of Experts (SEE) framework. SEE removes the need for an additional router, allowing each expert to independently decide whether a query should be handled. The framework employs distributed routing, and during continual fine-tuning, SEE only requires the training of new experts for incoming tasks rather than retraining the entire system. Experiments reveal that the SEE outperforms prior approaches, including multi-task learning, in continual fine-tuning. It also demonstrates remarkable generalization ability, as the expert can effectively identify out-of-distribution queries, which can then be directed to a more generalized model for resolution. This work highlights the promising potential of integrating routing and response mechanisms within each expert, paving the way for the future of distributed model ensembling.

Related papers

• Complexity Experts are Task-Discriminative Learners for Any Image Restoration [80.46313715427928]
  We introduce complexity experts" -- flexible expert blocks with varying computational complexity and receptive fields.<n>This preference effectively drives task-specific allocation, assigning tasks to experts with the appropriate complexity.<n>The proposed MoCE-IR model outperforms state-of-the-art methods, affirming its efficiency and practical applicability.
  arXiv  Detail & Related papers  (2024-11-27T15:58:07Z)
• Learning to Route Among Specialized Experts for Zero-Shot Generalization [39.56470042680907]
  We propose Post-Hoc Adaptive Tokenwise Gating Over an Ocean of Specialized Experts (PHATGOOSE) It learns to route among specialized modules that were produced through parameter-efficient fine-tuning. It does not require simultaneous access to the datasets used to create the specialized models and only requires a modest amount of additional compute after each expert model is trained.
  arXiv  Detail & Related papers  (2024-02-08T17:43:22Z)
• Divide and not forget: Ensemble of selectively trained experts in Continual Learning [0.2886273197127056]
  Class-incremental learning is becoming more popular as it helps models widen their applicability while not forgetting what they already know. A trend in this area is to use a mixture-of-expert technique, where different models work together to solve the task. SEED selects only one, the most optimal expert for a considered task, and uses data from this task to fine-tune only this expert.
  arXiv  Detail & Related papers  (2024-01-18T18:25:29Z)
• Inverse Reinforcement Learning with Sub-optimal Experts [56.553106680769474]
  We study the theoretical properties of the class of reward functions that are compatible with a given set of experts. Our results show that the presence of multiple sub-optimal experts can significantly shrink the set of compatible rewards. We analyze a uniform sampling algorithm that results in being minimax optimal whenever the sub-optimal experts' performance level is sufficiently close to the one of the optimal agent.
  arXiv  Detail & Related papers  (2024-01-08T12:39:25Z)
• Data-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning [85.66907881270785]
  We propose a data curriculum method, namely Data-CUBE, that arranges the orders of all the multi-task data for training. In the task level, we aim to find the optimal task order to minimize the total cross-task interference risk. In the instance level, we measure the difficulty of all instances per task, then divide them into the easy-to-difficult mini-batches for training.
  arXiv  Detail & Related papers  (2024-01-07T18:12:20Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)
• Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners [74.92558307689265]
  We propose Mod-Squad, a new model that is Modularized into groups of experts (a 'Squad') We optimize this matching process during the training of a single model. Experiments on the Taskonomy dataset with 13 vision tasks and the PASCAL-Context dataset with 5 vision tasks show the superiority of our approach.
  arXiv  Detail & Related papers  (2022-12-15T18:59:52Z)
• Diversified Dynamic Routing for Vision Tasks [36.199659460868496]
  We propose a novel architecture where each layer is composed of a set of experts. In our method, the model is explicitly trained to solve the challenge of finding relevant partitioning of the data. We conduct several experiments on semantic segmentation on Cityscapes and object detection and instance segmentation on MS-COCO.
  arXiv  Detail & Related papers  (2022-09-26T23:27:51Z)
• Generalizing to New Tasks via One-Shot Compositional Subgoals [23.15624959305799]
  The ability to generalize to previously unseen tasks with little to no supervision is a key challenge in modern machine learning research. We introduce CASE which attempts to address these issues by training an Imitation Learning agent using adaptive "near future" subgoals. Our experiments show that the proposed approach consistently outperforms the previous state-of-the-art compositional Imitation Learning approach by 30%.
  arXiv  Detail & Related papers  (2022-05-16T14:30:11Z)
• Hierarchical Few-Shot Imitation with Skill Transition Models [66.81252581083199]
  Few-shot Imitation with Skill Transition Models (FIST) is an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments.
  arXiv  Detail & Related papers  (2021-07-19T15:56:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.