Mitigating Background Shift in Class-Incremental Semantic Segmentation

• URL: http://arxiv.org/abs/2407.11859v1
• Date: Tue, 16 Jul 2024 15:44:37 GMT
• Title: Mitigating Background Shift in Class-Incremental Semantic Segmentation
• Authors: Gilhan Park, WonJun Moon, SuBeen Lee, Tae-Young Kim, Jae-Pil Heo,
• Abstract summary: Class-Incremental Semantic(CISS) aims to learn new classes without forgetting the old ones. We propose a background-class separation framework for CISS.
• Score: 18.604420743751643
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Class-Incremental Semantic Segmentation(CISS) aims to learn new classes without forgetting the old ones, using only the labels of the new classes. To achieve this, two popular strategies are employed: 1) pseudo-labeling and knowledge distillation to preserve prior knowledge; and 2) background weight transfer, which leverages the broad coverage of background in learning new classes by transferring background weight to the new class classifier. However, the first strategy heavily relies on the old model in detecting old classes while undetected pixels are regarded as the background, thereby leading to the background shift towards the old classes(i.e., misclassification of old class as background). Additionally, in the case of the second approach, initializing the new class classifier with background knowledge triggers a similar background shift issue, but towards the new classes. To address these issues, we propose a background-class separation framework for CISS. To begin with, selective pseudo-labeling and adaptive feature distillation are to distill only trustworthy past knowledge. On the other hand, we encourage the separation between the background and new classes with a novel orthogonal objective along with label-guided output distillation. Our state-of-the-art results validate the effectiveness of these proposed methods.

Related papers

• Early Preparation Pays Off: New Classifier Pre-tuning for Class Incremental Semantic Segmentation [13.62129805799111]
  Class incremental semantic segmentation aims to preserve old knowledge while learning new tasks. It is impeded by catastrophic forgetting and background shift issues. We propose a new classifier pre-tuning(NeST) method applied before the formal training process.
  arXiv  Detail & Related papers  (2024-07-19T09:19:29Z)
• PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning [49.240408681098906]
  Class-incremental learning (CIL) aims to recognize new classes incrementally while maintaining the discriminability of old classes. Most existing CIL methods are exemplar-based, i.e., storing a part of old data for retraining. We present a simple and novel dual bias reduction framework that employs self-supervised transformation (SST) in input space and prototype augmentation (protoAug) in deep feature space.
  arXiv  Detail & Related papers  (2024-07-19T05:03:16Z)
• Cs2K: Class-specific and Class-shared Knowledge Guidance for Incremental Semantic Segmentation [31.82132159867097]
  Incremental semantic segmentation endeavors to segment newly encountered classes while maintaining knowledge of old classes. We propose the Class-specific and Class-shared Knowledge (Cs2K) guidance for incremental semantic segmentation. Our proposed Cs2K significantly improves segmentation performance and is plug-and-play.
  arXiv  Detail & Related papers  (2024-07-12T07:15:26Z)
• Active Generalized Category Discovery [60.69060965936214]
  Generalized Category Discovery (GCD) endeavors to cluster unlabeled samples from both novel and old classes. We take the spirit of active learning and propose a new setting called Active Generalized Category Discovery (AGCD) Our method achieves state-of-the-art performance on both generic and fine-grained datasets.
  arXiv  Detail & Related papers  (2024-03-07T07:12:24Z)
• Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration [67.69532794049445]
  We find a tendency for existing methods to misclassify the samples of new classes into base classes, which leads to the poor performance of new classes. We propose a simple yet effective Training-frEE calibratioN (TEEN) strategy to enhance the discriminability of new classes.
  arXiv  Detail & Related papers  (2023-12-08T18:24:08Z)
• DiffusePast: Diffusion-based Generative Replay for Class Incremental Semantic Segmentation [73.54038780856554]
  Class Incremental Semantic (CISS) extends the traditional segmentation task by incrementally learning newly added classes. Previous work has introduced generative replay, which involves replaying old class samples generated from a pre-trained GAN. We propose DiffusePast, a novel framework featuring a diffusion-based generative replay module that generates semantically accurate images with more reliable masks guided by different instructions.
  arXiv  Detail & Related papers  (2023-08-02T13:13:18Z)
• Mining Unseen Classes via Regional Objectness: A Simple Baseline for Incremental Segmentation [57.80416375466496]
  Incremental or continual learning has been extensively studied for image classification tasks to alleviate catastrophic forgetting. We propose a simple yet effective method in this paper, named unseen Classes via Regional Objectness for Mining (MicroSeg) Our MicroSeg is based on the assumption that background regions with strong objectness possibly belong to those concepts in the historical or future stages. In this way, the distribution characterizes of old concepts in the feature space could be better perceived, relieving the catastrophic forgetting caused by the background shift accordingly.
  arXiv  Detail & Related papers  (2022-11-13T10:06:17Z)
• Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation [38.52441363934223]
  In class-incremental semantic segmentation (CISS), deep learning architectures suffer from the critical problems of catastrophic forgetting and semantic background shift. We propose a novel method which employs gradient-based attribution to identify the most relevant weights for new classes. Our experiments demonstrate significant improvement in mIoU compared to the state-of-the-art CISS methods on the Pascal-VOC 2012, ADE20K and Cityscapes datasets.
  arXiv  Detail & Related papers  (2022-10-13T17:32:12Z)
• Few-shot Open-set Recognition Using Background as Unknowns [58.04165813493666]
  Few-shot open-set recognition aims to classify both seen and novel images given only limited training data of seen classes. Our proposed method not only outperforms multiple baselines but also sets new results on three popular benchmarks.
  arXiv  Detail & Related papers  (2022-07-19T04:19:29Z)
• Memory-Free Generative Replay For Class-Incremental Learning [32.39857105540859]
  We propose a memory-free generative replay strategy to preserve fine-grained old classes characteristics. Our method is best complemented by prior regularization-based methods proved to be effective for easily distinguishable old classes.
  arXiv  Detail & Related papers  (2021-09-01T12:19:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.