Early Preparation Pays Off: New Classifier Pre-tuning for Class Incremental Semantic Segmentation

• URL: http://arxiv.org/abs/2407.14142v1
• Date: Fri, 19 Jul 2024 09:19:29 GMT
• Title: Early Preparation Pays Off: New Classifier Pre-tuning for Class Incremental Semantic Segmentation
• Authors: Zhengyuan Xie, Haiquan Lu, Jia-wen Xiao, Enguang Wang, Le Zhang, Xialei Liu,
• Abstract summary: Class incremental semantic segmentation aims to preserve old knowledge while learning new tasks. It is impeded by catastrophic forgetting and background shift issues. We propose a new classifier pre-tuning(NeST) method applied before the formal training process.
• Score: 13.62129805799111
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Class incremental semantic segmentation aims to preserve old knowledge while learning new tasks, however, it is impeded by catastrophic forgetting and background shift issues. Prior works indicate the pivotal importance of initializing new classifiers and mainly focus on transferring knowledge from the background classifier or preparing classifiers for future classes, neglecting the flexibility and variance of new classifiers. In this paper, we propose a new classifier pre-tuning~(NeST) method applied before the formal training process, learning a transformation from old classifiers to generate new classifiers for initialization rather than directly tuning the parameters of new classifiers. Our method can make new classifiers align with the backbone and adapt to the new data, preventing drastic changes in the feature extractor when learning new classes. Besides, we design a strategy considering the cross-task class similarity to initialize matrices used in the transformation, helping achieve the stability-plasticity trade-off. Experiments on Pascal VOC 2012 and ADE20K datasets show that the proposed strategy can significantly improve the performance of previous methods. The code is available at \url{https://github.com/zhengyuan-xie/ECCV24_NeST}.

Related papers

• PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning [49.240408681098906]
  Class-incremental learning (CIL) aims to recognize new classes incrementally while maintaining the discriminability of old classes. Most existing CIL methods are exemplar-based, i.e., storing a part of old data for retraining. We present a simple and novel dual bias reduction framework that employs self-supervised transformation (SST) in input space and prototype augmentation (protoAug) in deep feature space.
  arXiv  Detail & Related papers  (2024-07-19T05:03:16Z)
• Mitigating Background Shift in Class-Incremental Semantic Segmentation [18.604420743751643]
  Class-Incremental Semantic(CISS) aims to learn new classes without forgetting the old ones. We propose a background-class separation framework for CISS.
  arXiv  Detail & Related papers  (2024-07-16T15:44:37Z)
• Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning [65.57123249246358]
  We propose ExpAndable Subspace Ensemble (EASE) for PTM-based CIL. We train a distinct lightweight adapter module for each new task, aiming to create task-specific subspaces. Our prototype complement strategy synthesizes old classes' new features without using any old class instance.
  arXiv  Detail & Related papers  (2024-03-18T17:58:13Z)
• Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration [67.69532794049445]
  We find a tendency for existing methods to misclassify the samples of new classes into base classes, which leads to the poor performance of new classes. We propose a simple yet effective Training-frEE calibratioN (TEEN) strategy to enhance the discriminability of new classes.
  arXiv  Detail & Related papers  (2023-12-08T18:24:08Z)
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning [120.53458753007851]
  Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. We deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse. We propose a neural collapse inspired framework for FSCIL. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances.
  arXiv  Detail & Related papers  (2023-02-06T18:39:40Z)
• Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation [38.52441363934223]
  In class-incremental semantic segmentation (CISS), deep learning architectures suffer from the critical problems of catastrophic forgetting and semantic background shift. We propose a novel method which employs gradient-based attribution to identify the most relevant weights for new classes. Our experiments demonstrate significant improvement in mIoU compared to the state-of-the-art CISS methods on the Pascal-VOC 2012, ADE20K and Cityscapes datasets.
  arXiv  Detail & Related papers  (2022-10-13T17:32:12Z)
• New Insights on Reducing Abrupt Representation Change in Online Continual Learning [69.05515249097208]
  We focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream. We show that applying Experience Replay causes the newly added classes' representations to overlap significantly with the previous classes. We propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes.
  arXiv  Detail & Related papers  (2022-03-08T01:37:00Z)
• Self-Supervised Class Incremental Learning [51.62542103481908]
  Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels. When updating them based on the new class data, they suffer from catastrophic forgetting: the model cannot discern old class data clearly from the new. In this paper, we explore the performance of Self-Supervised representation learning in Class Incremental Learning (SSCIL) for the first time.
  arXiv  Detail & Related papers  (2021-11-18T06:58:19Z)
• Few-Shot Incremental Learning with Continually Evolved Classifiers [46.278573301326276]
  Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points. The difficulty lies in that limited data from new classes not only lead to significant overfitting issues but also exacerbate the notorious catastrophic forgetting problems. We propose a Continually Evolved CIF ( CEC) that employs a graph model to propagate context information between classifiers for adaptation.
  arXiv  Detail & Related papers  (2021-04-07T10:54:51Z)
• Class-incremental Learning with Pre-allocated Fixed Classifiers [20.74548175713497]
  In class-incremental learning, a learning agent faces a stream of data with the goal of learning new classes while not forgetting previous ones. We propose a novel fixed classifier in which a number of pre-allocated output nodes are subject to the classification loss right from the beginning of the learning phase.
  arXiv  Detail & Related papers  (2020-10-16T22:40:28Z)
• Modeling the Background for Incremental Learning in Semantic Segmentation [39.025848280224785]
  Deep architectures are vulnerable to catastrophic forgetting. This paper addresses this problem in the context of semantic segmentation. We propose a new distillation-based framework which explicitly accounts for this shift.
  arXiv  Detail & Related papers  (2020-02-03T13:30:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.