Causally Sufficient and Necessary Feature Expansion for Class-Incremental Learning

• URL: http://arxiv.org/abs/2603.09145v1
• Date: Tue, 10 Mar 2026 03:33:33 GMT
• Title: Causally Sufficient and Necessary Feature Expansion for Class-Incremental Learning
• Abstract summary: Current expansion-based methods for Class Incremental Learning (CIL) effectively catastrophic forgetting by freezing old features.<n> spurious feature correlations are the main cause of this collision, manifesting in two scopes.<n>We propose a Probability of Necessity and Sufficiency (PNS)-based regularization method to guide feature expansion in CIL.
• Score: 10.618608207545153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current expansion-based methods for Class Incremental Learning (CIL) effectively mitigate catastrophic forgetting by freezing old features. However, such task-specific features learned from the new task may collide with the old features. From a causal perspective, spurious feature correlations are the main cause of this collision, manifesting in two scopes: (i) guided by empirical risk minimization (ERM), intra-task spurious correlations cause task-specific features to rely on shortcut features. These non-robust features are vulnerable to interference, inevitably drifting into the feature space of other tasks; (ii) inter-task spurious correlations induce semantic confusion between visually similar classes across tasks. To address this, we propose a Probability of Necessity and Sufficiency (PNS)-based regularization method to guide feature expansion in CIL. Specifically, we first extend the definition of PNS to expansion-based CIL, termed CPNS, which quantifies both the causal completeness of intra-task representations and the separability of inter-task representations. We then introduce a dual-scope counterfactual generator based on twin networks to ensure the measurement of CPNS, which simultaneously generates: (i) intra-task counterfactual features to minimize intra-task PNS risk and ensure causal completeness of task-specific features, and (ii) inter-task interfering features to minimize inter-task PNS risk, ensuring the separability of inter-task representations. Theoretical analyses confirm its reliability. The regularization is a plug-and-play method for expansion-based CIL to mitigate feature collision. Extensive experiments demonstrate the effectiveness of the proposed method.

Related papers

• MechRL: Reinforcement Learning Agents Perform Circuit Discovery for Mechanistic Interpretability [0.0]
  We recast circuit discovery as a reinforcement-learning problem.<n>An agent operates over the 144 attention heads of GPT-2 small as a discrete action space.<n>A single PPO policy attains the per-episode oracle on both training tasks and on a held-out third task.
  arXiv  Detail & Related papers  (2026-05-25T21:32:57Z)
• Joint Interference Detection and Identification via Adversarial Multi-task Learning [0.0]
  We present the adversarial multi-task interference detection and identification network (AMTIDIN)<n>AMTIDIN integrates adversarial training to minimize distributional discrepancies across tasks and uses adaptive coefficients to model task correlations.<n>It significantly outperforms both its task-specific STL baseline and state-of-the-art MTL baselines in generalization and robustness.
  arXiv  Detail & Related papers  (2026-04-08T14:41:03Z)
• Model Merging in the Essential Subspace [78.5390284258307]
  Model merging aims to integrate multiple task-specific fine-tuned models into a single multi-task model without additional training.<n>Despite extensive research, task interference remains a major obstacle that often undermines the performance of merged models.<n>We propose ESM (Essential Subspace Merging), a robust framework for effective model merging.
  arXiv  Detail & Related papers  (2026-02-23T00:33:38Z)
• On the Paradoxical Interference between Instruction-Following and Task Solving [50.75960598434753]
  Instruction following aims to align Large Language Models (LLMs) with human intent by specifying explicit constraints on how tasks should be performed.<n>We reveal a counterintuitive phenomenon: instruction following can paradoxically interfere with LLMs' task-solving capability.<n>We propose a metric, SUSTAINSCORE, to quantify the interference of instruction following with task solving.
  arXiv  Detail & Related papers  (2026-01-29T17:48:56Z)
• FedGTEA: Federated Class-Incremental Learning with Gaussian Task Embedding and Alignment [4.031060983228927]
  We introduce a novel framework for Federated Class Incremental Learning, called Federated Gaussian Task Embedding and Alignment (FedGTEA)<n>FedGTEA is designed to capture task-specific knowledge and model uncertainty in a scalable and communication-efficient manner.
  arXiv  Detail & Related papers  (2025-10-14T19:02:41Z)
• MEJO: MLLM-Engaged Surgical Triplet Recognition via Inter- and Intra-Task Joint Optimization [52.149337961205624]
  We propose a framework that empowers both inter- and intra-task optimization for surgical triplet recognition.<n>For inter-task optimization, we introduce the Shared-Specific-Disentangled (S$2$D) learning scheme that decomposes representations into task-shared and task-specific components.<n>For intra-task optimization conflicts, we develop a Coordinated Gradient Learning (CGL) strategy, which dissects and rebalances the positive-negative ambiguities.
  arXiv  Detail & Related papers  (2025-09-16T09:48:52Z)
• Low-rank Prompt Interaction for Continual Vision-Language Retrieval [47.323830129786145]
  We propose the Low-rank Prompt Interaction to address the problem of multi-modal understanding.<n>Considering that the training parameters scale to the number of layers and tasks, we propose low-rank interaction-augmented decomposition.<n>We also adopt hierarchical low-rank contrastive learning to ensure robustness training.
  arXiv  Detail & Related papers  (2025-01-24T10:00:47Z)
• Task Indicating Transformer for Task-conditional Dense Predictions [16.92067246179703]
  We introduce a novel task-conditional framework called Task Indicating Transformer (TIT) to tackle this challenge. Our approach designs a Mix Task Adapter module within the transformer block, which incorporates a Task Indicating Matrix through matrix decomposition. We also propose a Task Gate Decoder module that harnesses a Task Indicating Vector and gating mechanism to facilitate adaptive multi-scale feature refinement.
  arXiv  Detail & Related papers  (2024-03-01T07:06:57Z)
• A Unified Causal View of Instruction Tuning [76.1000380429553]
  We develop a meta Structural Causal Model (meta-SCM) to integrate different NLP tasks under a single causal structure of the data. Key idea is to learn task-required causal factors and only use those to make predictions for a given task.
  arXiv  Detail & Related papers  (2024-02-09T07:12:56Z)
• Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives [19.90788777476128]
  Multi-task learning (MTL) seeks to learn a single model to accomplish multiple tasks by leveraging shared information among the tasks. Existing MTL models have been known to suffer from negative interference among tasks. We propose ETR-NLP to mitigate task interference through a synergistic combination of non-learnable primitives and explicit task routing.
  arXiv  Detail & Related papers  (2023-08-03T22:34:16Z)
• Dense Network Expansion for Class Incremental Learning [61.00081795200547]
  State-of-the-art approaches use a dynamic architecture based on network expansion (NE), in which a task expert is added per task. A new NE method, dense network expansion (DNE), is proposed to achieve a better trade-off between accuracy and model complexity. It outperforms the previous SOTA methods by a margin of 4% in terms of accuracy, with similar or even smaller model scale.
  arXiv  Detail & Related papers  (2023-03-22T16:42:26Z)
• Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference [75.95287293847697]
  Two common challenges in developing multi-task models are often overlooked in literature. First, enabling the model to be inherently incremental, continuously incorporating information from new tasks without forgetting the previously learned ones (incremental learning) Second, eliminating adverse interactions amongst tasks, which has been shown to significantly degrade the single-task performance in a multi-task setup (task interference)
  arXiv  Detail & Related papers  (2020-07-24T14:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.