Modeling the Background for Incremental Learning in Semantic Segmentation

• URL: http://arxiv.org/abs/2002.00718v2
• Date: Mon, 30 Mar 2020 14:01:26 GMT
• Title: Modeling the Background for Incremental Learning in Semantic Segmentation
• Authors: Fabio Cermelli, Massimiliano Mancini, Samuel Rota Bul\`o, Elisa Ricci, Barbara Caputo
• Abstract summary: Deep architectures are vulnerable to catastrophic forgetting. This paper addresses this problem in the context of semantic segmentation. We propose a new distillation-based framework which explicitly accounts for this shift.
• Score: 39.025848280224785
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite their effectiveness in a wide range of tasks, deep architectures suffer from some important limitations. In particular, they are vulnerable to catastrophic forgetting, i.e. they perform poorly when they are required to update their model as new classes are available but the original training set is not retained. This paper addresses this problem in the context of semantic segmentation. Current strategies fail on this task because they do not consider a peculiar aspect of semantic segmentation: since each training step provides annotation only for a subset of all possible classes, pixels of the background class (i.e. pixels that do not belong to any other classes) exhibit a semantic distribution shift. In this work we revisit classical incremental learning methods, proposing a new distillation-based framework which explicitly accounts for this shift. Furthermore, we introduce a novel strategy to initialize classifier's parameters, thus preventing biased predictions toward the background class. We demonstrate the effectiveness of our approach with an extensive evaluation on the Pascal-VOC 2012 and ADE20K datasets, significantly outperforming state of the art incremental learning methods.

Related papers

• Early Preparation Pays Off: New Classifier Pre-tuning for Class Incremental Semantic Segmentation [13.62129805799111]
  Class incremental semantic segmentation aims to preserve old knowledge while learning new tasks. It is impeded by catastrophic forgetting and background shift issues. We propose a new classifier pre-tuning(NeST) method applied before the formal training process.
  arXiv  Detail & Related papers  (2024-07-19T09:19:29Z)
• ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning [54.68180752416519]
  Panoptic segmentation is a cutting-edge computer vision task. We introduce a novel and efficient method for continual panoptic segmentation based on Visual Prompt Tuning, dubbed ECLIPSE. Our approach involves freezing the base model parameters and fine-tuning only a small set of prompt embeddings, addressing both catastrophic forgetting and plasticity.
  arXiv  Detail & Related papers  (2024-03-29T11:31:12Z)
• Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration [67.69532794049445]
  We find a tendency for existing methods to misclassify the samples of new classes into base classes, which leads to the poor performance of new classes. We propose a simple yet effective Training-frEE calibratioN (TEEN) strategy to enhance the discriminability of new classes.
  arXiv  Detail & Related papers  (2023-12-08T18:24:08Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• Evidential Deep Learning for Class-Incremental Semantic Segmentation [15.563703446465823]
  Class-Incremental Learning is a challenging problem in machine learning that aims to extend previously trained neural networks with new classes. In this paper, we address the problem of how to model unlabeled classes while avoiding spurious feature clustering of future uncorrelated classes. Our method factorizes the problem into a separate foreground class probability, calculated by the expected value of the Dirichlet distribution, and an unknown class (background) probability corresponding to the uncertainty of the estimate.
  arXiv  Detail & Related papers  (2022-12-06T10:13:30Z)
• Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation [39.025848280224785]
  We introduce a novel incremental class learning approach for semantic segmentation. Since each training step provides annotation only for a subset of all possible classes, pixels of the background class exhibit a semantic shift. We demonstrate the effectiveness of our approach with an extensive evaluation on the Pascal-VOC, ADE20K, and Cityscapes datasets.
  arXiv  Detail & Related papers  (2022-01-31T16:33:21Z)
• Tackling Catastrophic Forgetting and Background Shift in Continual Semantic Segmentation [35.2461834832935]
  Continual learning for semantic segmentation (CSS) is an emerging trend that consists in updating an old model by sequentially adding new classes. In this paper, we propose Local POD, a multi-scale pooling distillation scheme that preserves long- and short-range spatial relationships. We also introduce a novel rehearsal method that is particularly suited for segmentation.
  arXiv  Detail & Related papers  (2021-06-29T11:57:21Z)
• A Few Guidelines for Incremental Few-Shot Segmentation [57.34237650765928]
  Given a pretrained segmentation model and few images containing novel classes, our goal is to learn to segment novel classes while retaining the ability to segment previously seen ones. We show how the main problems of end-to-end training in this scenario are. i) the drift of the batch-normalization statistics toward novel classes that we can fix with batch renormalization and. ii) the forgetting of old classes, that we can fix with regularization strategies.
  arXiv  Detail & Related papers  (2020-11-30T20:45:56Z)
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
  State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results. Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
  arXiv  Detail & Related papers  (2020-08-04T10:41:32Z)
• Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels [35.586031601299034]
  We present a technique implementing class-incremental learning for semantic segmentation without using the labeled data the model was initially trained on. We show how to overcome these problems with a novel class-incremental learning technique, which nonetheless requires labels only for the new classes. We evaluate our method on the Cityscapes dataset, where we exceed the mIoU performance of all baselines by 3.5% absolute reaching a result.
  arXiv  Detail & Related papers  (2020-05-12T21:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.