Gradient-Semantic Compensation for Incremental Semantic Segmentation

• URL: http://arxiv.org/abs/2307.10822v1
• Date: Thu, 20 Jul 2023 12:32:25 GMT
• Title: Gradient-Semantic Compensation for Incremental Semantic Segmentation
• Authors: Wei Cong, Yang Cong, Jiahua Dong, Gan Sun, Henghui Ding
• Abstract summary: Incremental semantic segmentation aims to continually learn the segmentation of new coming classes without accessing the training data of previously learned classes. We propose a Gradient-Semantic Compensation model, which surmounts incremental semantic segmentation from both gradient and semantic perspectives.
• Score: 43.00965727428193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Incremental semantic segmentation aims to continually learn the segmentation of new coming classes without accessing the training data of previously learned classes. However, most current methods fail to address catastrophic forgetting and background shift since they 1) treat all previous classes equally without considering different forgetting paces caused by imbalanced gradient back-propagation; 2) lack strong semantic guidance between classes. To tackle the above challenges, in this paper, we propose a Gradient-Semantic Compensation (GSC) model, which surmounts incremental semantic segmentation from both gradient and semantic perspectives. Specifically, to address catastrophic forgetting from the gradient aspect, we develop a step-aware gradient compensation that can balance forgetting paces of previously seen classes via re-weighting gradient backpropagation. Meanwhile, we propose a soft-sharp semantic relation distillation to distill consistent inter-class semantic relations via soft labels for alleviating catastrophic forgetting from the semantic aspect. In addition, we develop a prototypical pseudo re-labeling that provides strong semantic guidance to mitigate background shift. It produces high-quality pseudo labels for old classes in the background by measuring distances between pixels and class-wise prototypes. Extensive experiments on three public datasets, i.e., Pascal VOC 2012, ADE20K, and Cityscapes, demonstrate the effectiveness of our proposed GSC model.

Related papers

• Physically Feasible Semantic Segmentation [58.17907376475596]
  State-of-the-art semantic segmentation models are typically optimized in a data-driven fashion. Our method, Physically Feasible Semantic (PhyFea), extracts explicit physical constraints that govern spatial class relations. PhyFea yields significant performance improvements in mIoU over each state-of-the-art network we use.
  arXiv  Detail & Related papers  (2024-08-26T22:39:08Z)
• Class Gradient Projection For Continual Learning [99.105266615448]
  Catastrophic forgetting is one of the most critical challenges in Continual Learning (CL) We propose Class Gradient Projection (CGP), which calculates the gradient subspace from individual classes rather than tasks.
  arXiv  Detail & Related papers  (2023-11-25T02:45:56Z)
• Inherit with Distillation and Evolve with Contrast: Exploring Class Incremental Semantic Segmentation Without Exemplar Memory [23.730424035141155]
  We present IDEC, which consists of a Dense Knowledge Distillation on all Aspects (DADA) and an Asymmetric Region-wise Contrastive Learning (ARCL) module. We demonstrate the effectiveness of our method on multiple CISS tasks by state-of-the-art performance, including Pascal VOC 2012, ADE20K and ISPRS datasets.
  arXiv  Detail & Related papers  (2023-09-27T05:38:31Z)
• Advancing Incremental Few-shot Semantic Segmentation via Semantic-guided Relation Alignment and Adaptation [98.51938442785179]
  Incremental few-shot semantic segmentation aims to incrementally extend a semantic segmentation model to novel classes. This task faces a severe semantic-aliasing issue between base and novel classes due to data imbalance. We propose the Semantic-guided Relation Alignment and Adaptation (SRAA) method that fully considers the guidance of prior semantic information.
  arXiv  Detail & Related papers  (2023-05-18T10:40:52Z)
• Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation [39.025848280224785]
  We introduce a novel incremental class learning approach for semantic segmentation. Since each training step provides annotation only for a subset of all possible classes, pixels of the background class exhibit a semantic shift. We demonstrate the effectiveness of our approach with an extensive evaluation on the Pascal-VOC, ADE20K, and Cityscapes datasets.
  arXiv  Detail & Related papers  (2022-01-31T16:33:21Z)
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation [35.2461834832935]
  Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships. We also introduce a novel rehearsal method that is particularly suited for segmentation.
  arXiv  Detail & Related papers  (2021-06-29T11:57:21Z)
• Anti-aliasing Semantic Reconstruction for Few-Shot Semantic Segmentation [66.85202434812942]
  We reformulate few-shot segmentation as a semantic reconstruction problem. We convert base class features into a series of basis vectors which span a class-level semantic space for novel class reconstruction. Our proposed approach, referred to as anti-aliasing semantic reconstruction (ASR), provides a systematic yet interpretable solution for few-shot learning problems.
  arXiv  Detail & Related papers  (2021-06-01T02:17:36Z)
• A Simple Baseline for Semi-supervised Semantic Segmentation with Strong Data Augmentation [74.8791451327354]
  We propose a simple yet effective semi-supervised learning framework for semantic segmentation. A set of simple design and training techniques can collectively improve the performance of semi-supervised semantic segmentation significantly. Our method achieves state-of-the-art results in the semi-supervised settings on the Cityscapes and Pascal VOC datasets.
  arXiv  Detail & Related papers  (2021-04-15T06:01:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.