Quantum-Gated Task-interaction Knowledge Distillation for Pre-trained Model-based Class-Incremental Learning

• URL: http://arxiv.org/abs/2604.11112v1
• Date: Mon, 13 Apr 2026 07:28:44 GMT
• Title: Quantum-Gated Task-interaction Knowledge Distillation for Pre-trained Model-based Class-Incremental Learning
• Abstract summary: Class-incremental learning (CIL) aims to continuously accumulate knowledge from a stream of tasks.<n>CIL still struggles with the entanglement of multi-task subspaces.<n>We propose a novel Quantum-Gated Task-interaction Knowledge Distillation framework.
• Score: 31.28583279686815
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Class-incremental learning (CIL) aims to continuously accumulate knowledge from a stream of tasks and construct a unified classifier over all seen classes. Although pretrained models (PTMs) have shown promising performance in CIL, they still struggle with the entanglement of multi-task subspaces, leading to catastrophic forgetting when task routing parameters are poorly calibrated or task-level representations are rigidly fixed. To address this issue, we propose a novel Quantum-Gated Task-interaction Knowledge Distillation (QKD) framework that leverages quantum gating to guide inter-task knowledge transfer. Specifically, we introduce a quantum-gated task modulation gating mechanism to model the relational dependencies among task embedding, dynamically capturing the sample-to-task relevance for both joint training and inference across streaming tasks. Guided by the quantum gating outputs, we perform task-interaction knowledge distillation guided by these task-embedding-level correlation weights from old to new adapters, enabling the model to bridge the representation gaps between independent task subspaces. Extensive experiments demonstrate that QKD effectively mitigates forgetting and achieves state-of-the-art performance.

Related papers

• On the Paradoxical Interference between Instruction-Following and Task Solving [50.75960598434753]
  Instruction following aims to align Large Language Models (LLMs) with human intent by specifying explicit constraints on how tasks should be performed.<n>We reveal a counterintuitive phenomenon: instruction following can paradoxically interfere with LLMs' task-solving capability.<n>We propose a metric, SUSTAINSCORE, to quantify the interference of instruction following with task solving.
  arXiv  Detail & Related papers  (2026-01-29T17:48:56Z)
• Purifying Task Vectors in Knowledge-Aware Subspace for Model Merging [83.5273168208788]
  Model merging aims to integrate task-specific abilities from individually fine-tuned models into a single model without extra training.<n>The merged model often suffers from notable performance degradation due to the conflicts caused by task-irrelevant redundancy in task vectors.<n>We propose Purifying TAsk Vectors (PAVE) in knowledge-aware subspace to overcome these challenges.
  arXiv  Detail & Related papers  (2025-10-16T14:02:57Z)
• Quantum-Enhanced Multi-Task Learning with Learnable Weighting for Pharmacokinetic and Toxicity Prediction [10.487649921110611]
  We propose a new unified Quantum-enhanced and task-Weighted Multi-Task Learning framework, specifically designed for ADMET classification tasks.<n>QW-MTL adopts quantum chemical descriptors to enrich molecular representations with additional information about the electronic structure and interactions.<n>It introduces a novel exponential task weighting scheme that combines dataset-scale priors with learnable parameters to achieve dynamic loss balancing across tasks.
  arXiv  Detail & Related papers  (2025-09-04T18:33:40Z)
• Efficient Knowledge Transfer in Multi-Task Learning through Task-Adaptive Low-Rank Representation [11.955971931186006]
  Pre-trained language models struggle with emerging tasks unseen during training in real-world applications.<n>We propose Task-Adaptive Low-Rank Representation (TA-LoRA), an MTL method built on prompt tuning.<n>Experiments on 16 tasks demonstrate that TA-LoRA achieves state-of-the-art performance in full-data and few-shot settings.
  arXiv  Detail & Related papers  (2025-04-20T06:33:19Z)
• Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge [12.367471198090655]
  Task Arithmetic (TA), which combines task vectors derived from fine-tuning, enables multi-task learning and task forgetting but struggles to isolate task-specific knowledge from general instruction-following behavior.<n>We propose Layer-Aware Task Arithmetic (LATA), a novel approach that assigns layer-specific weights to task vectors based on their alignment with instruction-following or task-specific components.
  arXiv  Detail & Related papers  (2025-02-27T15:22:14Z)
• Exploiting Task Relationships for Continual Learning Using Transferability-Aware Task Embeddings [8.814732457885022]
  Continual learning (CL) has been a critical topic in contemporary deep neural network applications.<n>We propose a transferability-aware task embedding, termed H-embedding, and construct a hypernet framework under its guidance.
  arXiv  Detail & Related papers  (2025-02-17T09:52:19Z)
• Mitigating Interference in the Knowledge Continuum through Attention-Guided Incremental Learning [17.236861687708096]
  Attention-Guided Incremental Learning' (AGILE) is a rehearsal-based CL approach that incorporates compact task attention to effectively reduce interference between tasks. AGILE significantly improves generalization performance by mitigating task interference and outperforming rehearsal-based approaches in several CL scenarios.
  arXiv  Detail & Related papers  (2024-05-22T20:29:15Z)
• PEMT: Multi-Task Correlation Guided Mixture-of-Experts Enables Parameter-Efficient Transfer Learning [28.353530290015794]
  We propose PEMT, a novel parameter-efficient fine-tuning framework based on multi-task transfer learning. We conduct experiments on a broad range of tasks over 17 datasets.
  arXiv  Detail & Related papers  (2024-02-23T03:59:18Z)
• Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning [56.26889258704261]
  We propose a novel "disentangled" decision-making method, Semantically Aligned task decomposition in MARL (SAMA) SAMA prompts pretrained language models with chain-of-thought that can suggest potential goals, provide suitable goal decomposition and subgoal allocation as well as self-reflection-based replanning. SAMA demonstrates considerable advantages in sample efficiency compared to state-of-the-art ASG methods.
  arXiv  Detail & Related papers  (2023-05-18T10:37:54Z)
• Composite Learning for Robust and Effective Dense Predictions [81.2055761433725]
  Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task. We find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks.
  arXiv  Detail & Related papers  (2022-10-13T17:59:16Z)
• Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism [120.1998866178014]
  We present a flexible framework for continual object detection via pRotOtypical taSk corrElaTion guided gaTingAnism (ROSETTA) Concretely, a unified framework is shared by all tasks while task-aware gates are introduced to automatically select sub-models for specific tasks. Experiments on COCO-VOC, KITTI-Kitchen, class-incremental detection on VOC and sequential learning of four tasks show that ROSETTA yields state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-06T07:31:28Z)
• Combining Modular Skills in Multitask Learning [149.8001096811708]
  A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. In this work, we assume each task is associated with a subset of latent discrete skills from a (potentially small) inventory. We find that the modular design of a network significantly increases sample efficiency in reinforcement learning and few-shot generalisation in supervised learning.
  arXiv  Detail & Related papers  (2022-02-28T16:07:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.