Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation

• URL: http://arxiv.org/abs/2104.00875v1
• Date: Fri, 2 Apr 2021 03:47:16 GMT
• Title: Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation
• Authors: Zilong Huang, Wentian Hao, Xinggang Wang, Mingyuan Tao, Jianqiang Huang, Wenyu Liu, Xian-Sheng Hua
• Abstract summary: convolutional neural networks are ill-equipped for incremental learning. New classes are available but the initial training data is not retained. We try to address this issue by "inverting" the trained segmentation network to synthesize input images starting from random noise.
• Score: 84.1985497426083
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Despite their success for semantic segmentation, convolutional neural networks are ill-equipped for incremental learning, \ie, adapting the original segmentation model as new classes are available but the initial training data is not retained. Actually, they are vulnerable to catastrophic forgetting problem. We try to address this issue by "inverting" the trained segmentation network to synthesize input images starting from random noise. To avoid setting detailed pixel-wise segmentation maps as the supervision manually, we propose the SegInversion to synthesize images using the image-level labels. To increase the diversity of synthetic images, the Scale-Aware Aggregation module is integrated into SegInversion for controlling the scale (the number of pixels) of synthetic objects. Along with real images of new classes, the synthesized images will be fed into the distillation-based framework to train the new segmentation model which retains the information about previously learned classes, whilst updating the current model to learn the new ones. The proposed method significantly outperforms other incremental learning methods and obtains state-of-the-art performance on the PASCAL VOC 2012 and ADE20K datasets. The code and models will be made publicly available.

Related papers

• UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation [64.01742988773745]
  An increasing privacy concern exists regarding training large-scale image segmentation models on unauthorized private data. We exploit the concept of unlearnable examples to make images unusable to model training by generating and adding unlearnable noise into the original images. We empirically verify the effectiveness of UnSeg across 6 mainstream image segmentation tasks, 10 widely used datasets, and 7 different network architectures.
  arXiv  Detail & Related papers  (2024-10-13T16:34:46Z)
• RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation [28.02204928717511]
  We propose a weakly supervised approach to transfer objectness prior from the previously learned classes into the new ones. We show how even a simple pairwise interaction between classes can significantly improve the segmentation mask quality of both old and new classes.
  arXiv  Detail & Related papers  (2023-05-31T14:14:21Z)
• Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training of Image Segmentation Models [54.49581189337848]
  We propose a method to enable the end-to-end pre-training for image segmentation models based on classification datasets. The proposed method leverages a weighted segmentation learning procedure to pre-train the segmentation network en masse. Experiment results show that, with ImageNet accompanied by PSSL as the source dataset, the proposed end-to-end pre-training strategy successfully boosts the performance of various segmentation models.
  arXiv  Detail & Related papers  (2022-07-04T13:02:32Z)
• Incremental Learning in Semantic Segmentation from Image Labels [18.404068463921426]
  Existing semantic segmentation approaches achieve impressive results, but struggle to update their models incrementally as new categories are uncovered. This paper proposes a novel framework for Weakly Incremental Learning for Semantics, that aims at learning to segment new classes from cheap and largely available image-level labels. As opposed to existing approaches, that need to generate pseudo-labels offline, we use an auxiliary classifier, trained with image-level labels and regularized by the segmentation model, to obtain pseudo-supervision online and update the model incrementally.
  arXiv  Detail & Related papers  (2021-12-03T12:47:12Z)
• Segmenter: Transformer for Semantic Segmentation [79.9887988699159]
  We introduce Segmenter, a transformer model for semantic segmentation. We build on the recent Vision Transformer (ViT) and extend it to semantic segmentation. It outperforms the state of the art on the challenging ADE20K dataset and performs on-par on Pascal Context and Cityscapes.
  arXiv  Detail & Related papers  (2021-05-12T13:01:44Z)
• A Closer Look at Self-training for Zero-Label Semantic Segmentation [53.4488444382874]
  Being able to segment unseen classes not observed during training is an important technical challenge in deep learning. Prior zero-label semantic segmentation works approach this task by learning visual-semantic embeddings or generative models. We propose a consistency regularizer to filter out noisy pseudo-labels by taking the intersections of the pseudo-labels generated from different augmentations of the same image.
  arXiv  Detail & Related papers  (2021-04-21T14:34:33Z)
• Recursive Training for Zero-Shot Semantic Segmentation [26.89352005206994]
  We propose a training scheme to supervise the retraining of a semantic segmentation model for a zero-shot setting. We show that our proposed model achieves state-of-the-art performance on the Pascal-VOC 2012 dataset and Pascal-Context dataset.
  arXiv  Detail & Related papers  (2021-02-26T23:44:16Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.