Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning

• URL: http://arxiv.org/abs/2503.05423v1
• Date: Fri, 07 Mar 2025 13:50:29 GMT
• Title: Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
• Authors: Run He, Di Fang, Yicheng Xu, Yawen Cui, Ming Li, Cen Chen, Ziqian Zeng, Huiping Zhuang,
• Abstract summary: We propose a Dual-Projection Shift Estimation and Incremental Reconstruction (DPCR) approach for Exemplar-Free Class-Learning (EFCIL)<n>DPCR effectively estimates semantic shift through a dual-projection, which combines a row-space projection to capture both taskwise and categorywise shifts.<n>We demonstrate that, across various datasets, DPCR effectively balances old and new tasks, outperforming state-the-art EFCIL methods.
• Score: 20.581215770655383
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exemplar-Free Class-Incremental Learning (EFCIL) aims to sequentially learn from distinct categories without retaining exemplars but easily suffers from catastrophic forgetting of learned knowledge. While existing EFCIL methods leverage knowledge distillation to alleviate forgetting, they still face two critical challenges: semantic shift and decision bias. Specifically, the embeddings of old tasks shift in the embedding space after learning new tasks, and the classifier becomes biased towards new tasks due to training solely with new data, thereby hindering the balance between old and new knowledge. To address these issues, we propose the Dual-Projection Shift Estimation and Classifier Reconstruction (DPCR) approach for EFCIL. DPCR effectively estimates semantic shift through a dual-projection, which combines a learnable transformation with a row-space projection to capture both task-wise and category-wise shifts. Furthermore, to mitigate decision bias, DPCR employs ridge regression to reformulate classifier training as a reconstruction process. This reconstruction exploits previous information encoded in covariance and prototype of each class after calibration with estimated shift, thereby reducing decision bias. Extensive experiments demonstrate that, across various datasets, DPCR effectively balances old and new tasks, outperforming state-of-the-art EFCIL methods.

Related papers

• CoMBO: Conflict Mitigation via Branched Optimization for Class Incremental Segmentation [75.35841972192684]
  Effective Class Incremental (CIS) requires simultaneously mitigating catastrophic forgetting and ensuring sufficient plasticity to integrate new classes. We introduce a novel approach, Conflict Mitigation via Branched Optimization(CoMBO) Within this approach, we present the Query Conflict Reduction module, designed to explicitly refine queries for new classes.
  arXiv  Detail & Related papers  (2025-04-05T12:34:51Z)
• PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning [49.240408681098906]
  Class-incremental learning (CIL) aims to recognize new classes incrementally while maintaining the discriminability of old classes. Most existing CIL methods are exemplar-based, i.e., storing a part of old data for retraining. We present a simple and novel dual bias reduction framework that employs self-supervised transformation (SST) in input space and prototype augmentation (protoAug) in deep feature space.
  arXiv  Detail & Related papers  (2024-07-19T05:03:16Z)
• Adaptive Retention & Correction: Test-Time Training for Continual Learning [114.5656325514408]
  A common problem in continual learning is the classification layer's bias towards the most recent task.<n>We name our approach Adaptive Retention & Correction (ARC)<n>ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets.
  arXiv  Detail & Related papers  (2024-05-23T08:43:09Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Fixed Random Classifier Rearrangement for Continual Learning [0.5439020425819]
  In visual classification scenario, neural networks inevitably forget the knowledge of old tasks after learning new ones. We propose a continual learning algorithm named Fixed Random Rearrangement (FRCR)
  arXiv  Detail & Related papers  (2024-02-23T09:43:58Z)
• Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning [40.13416912075668]
  We propose a rehearsal-free CIL approach that learns continually via the synergy between two Complementary Learning Subnetworks. Our method achieves competitive results against state-of-the-art methods, especially in accuracy gain, memory cost, training efficiency, and task-order.
  arXiv  Detail & Related papers  (2023-06-21T01:43:25Z)
• Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion [50.03041373044267]
  We propose a Supervised Contrastive learning framework with adaptive classification criterion for Continual Learning. Experiments show that CFL achieves state-of-the-art performance and has a stronger ability to overcome compared with the classification baselines.
  arXiv  Detail & Related papers  (2023-05-20T19:22:40Z)
• Informative regularization for a multi-layer perceptron RR Lyrae classifier under data shift [3.303002683812084]
  We propose a scalable and easily adaptable approach based on an informative regularization and an ad-hoc training procedure to mitigate the shift problem. Our method provides a new path to incorporate knowledge from characteristic features into artificial neural networks to manage the underlying data shift problem.
  arXiv  Detail & Related papers  (2023-03-12T02:49:19Z)
• Prototypical quadruplet for few-shot class incremental learning [24.814045065163135]
  We propose a novel method that improves classification robustness by identifying a better embedding space using an improved contrasting loss. Our approach retains previously acquired knowledge in the embedding space, even when trained with new classes. We demonstrate the effectiveness of our method by showing that the embedding space remains intact after training the model with new classes and outperforms existing state-of-the-art algorithms in terms of accuracy across different sessions.
  arXiv  Detail & Related papers  (2022-11-05T17:19:14Z)
• Self-Supervised Class Incremental Learning [51.62542103481908]
  Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels. When updating them based on the new class data, they suffer from catastrophic forgetting: the model cannot discern old class data clearly from the new. In this paper, we explore the performance of Self-Supervised representation learning in Class Incremental Learning (SSCIL) for the first time.
  arXiv  Detail & Related papers  (2021-11-18T06:58:19Z)
• IB-DRR: Incremental Learning with Information-Back Discrete Representation Replay [4.8666876477091865]
  Incremental learning aims to enable machine learning models to continuously acquire new knowledge given new classes. Saving a subset of training samples of previously seen classes in the memory and replaying them during new training phases is proven to be an efficient and effective way to fulfil this aim. However, finding a trade-off between the model performance and the number of samples to save for each class is still an open problem for replay-based incremental learning.
  arXiv  Detail & Related papers  (2021-04-21T15:32:11Z)
• Incremental Embedding Learning via Zero-Shot Translation [65.94349068508863]
  Current state-of-the-art incremental learning methods tackle catastrophic forgetting problem in traditional classification networks. We propose a novel class-incremental method for embedding network, named as zero-shot translation class-incremental method (ZSTCI) In addition, ZSTCI can easily be combined with existing regularization-based incremental learning methods to further improve performance of embedding networks.
  arXiv  Detail & Related papers  (2020-12-31T08:21:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.