RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation

• URL: http://arxiv.org/abs/2305.19879v1
• Date: Wed, 31 May 2023 14:14:21 GMT
• Title: RaSP: Relation-aware Semantic Prior for Weakly Supervised Incremental Segmentation
• Authors: Subhankar Roy, Riccardo Volpi, Gabriela Csurka, Diane Larlus
• Abstract summary: We propose a weakly supervised approach to transfer objectness prior from the previously learned classes into the new ones. We show how even a simple pairwise interaction between classes can significantly improve the segmentation mask quality of both old and new classes.
• Score: 28.02204928717511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Class-incremental semantic image segmentation assumes multiple model updates, each enriching the model to segment new categories. This is typically carried out by providing expensive pixel-level annotations to the training algorithm for all new objects, limiting the adoption of such methods in practical applications. Approaches that solely require image-level labels offer an attractive alternative, yet, such coarse annotations lack precise information about the location and boundary of the new objects. In this paper we argue that, since classes represent not just indices but semantic entities, the conceptual relationships between them can provide valuable information that should be leveraged. We propose a weakly supervised approach that exploits such semantic relations to transfer objectness prior from the previously learned classes into the new ones, complementing the supervisory signal from image-level labels. We validate our approach on a number of continual learning tasks, and show how even a simple pairwise interaction between classes can significantly improve the segmentation mask quality of both old and new classes. We show these conclusions still hold for longer and, hence, more realistic sequences of tasks and for a challenging few-shot scenario.

Related papers

• Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation [56.1776710527814]
  Weakly Incremental Learning for Semantic (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-level labels. A prevailing way to solve WILSS is the generation of seed areas for each new class, serving as a form of pixel-level supervision. We propose an innovative, tendency-driven relationship of mutual exclusivity, meticulously tailored to govern the behavior of the seed areas.
  arXiv  Detail & Related papers  (2024-04-18T08:23:24Z)
• Class Enhancement Losses with Pseudo Labels for Zero-shot Semantic Segmentation [40.09476732999614]
  Mask proposal models have significantly improved the performance of zero-shot semantic segmentation. The use of a background' embedding during training in these methods is problematic as the resulting model tends to over-learn and assign all unseen classes as the background class instead of their correct labels. This paper proposes novel class enhancement losses to bypass the use of the background embbedding during training, and simultaneously exploit the semantic relationship between text embeddings and mask proposals by ranking the similarity scores.
  arXiv  Detail & Related papers  (2023-01-18T06:55:02Z)
• Modeling the Background for Incremental and Weakly-Supervised Semantic Segmentation [39.025848280224785]
  We introduce a novel incremental class learning approach for semantic segmentation. Since each training step provides annotation only for a subset of all possible classes, pixels of the background class exhibit a semantic shift. We demonstrate the effectiveness of our approach with an extensive evaluation on the Pascal-VOC, ADE20K, and Cityscapes datasets.
  arXiv  Detail & Related papers  (2022-01-31T16:33:21Z)
• Novel Class Discovery in Semantic Segmentation [104.30729847367104]
  We introduce a new setting of Novel Class Discovery in Semantic (NCDSS) It aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image. We propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels.
  arXiv  Detail & Related papers  (2021-12-03T13:31:59Z)
• Incremental Learning in Semantic Segmentation from Image Labels [18.404068463921426]
  Existing semantic segmentation approaches achieve impressive results, but struggle to update their models incrementally as new categories are uncovered. This paper proposes a novel framework for Weakly Incremental Learning for Semantics, that aims at learning to segment new classes from cheap and largely available image-level labels. As opposed to existing approaches, that need to generate pseudo-labels offline, we use an auxiliary classifier, trained with image-level labels and regularized by the segmentation model, to obtain pseudo-supervision online and update the model incrementally.
  arXiv  Detail & Related papers  (2021-12-03T12:47:12Z)
• A Closer Look at Self-training for Zero-Label Semantic Segmentation [53.4488444382874]
  Being able to segment unseen classes not observed during training is an important technical challenge in deep learning. Prior zero-label semantic segmentation works approach this task by learning visual-semantic embeddings or generative models. We propose a consistency regularizer to filter out noisy pseudo-labels by taking the intersections of the pseudo-labels generated from different augmentations of the same image.
  arXiv  Detail & Related papers  (2021-04-21T14:34:33Z)
• Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation [84.1985497426083]
  convolutional neural networks are ill-equipped for incremental learning. New classes are available but the initial training data is not retained. We try to address this issue by "inverting" the trained segmentation network to synthesize input images starting from random noise.
  arXiv  Detail & Related papers  (2021-04-02T03:47:16Z)
• A Few Guidelines for Incremental Few-Shot Segmentation [57.34237650765928]
  Given a pretrained segmentation model and few images containing novel classes, our goal is to learn to segment novel classes while retaining the ability to segment previously seen ones. We show how the main problems of end-to-end training in this scenario are. i) the drift of the batch-normalization statistics toward novel classes that we can fix with batch renormalization and. ii) the forgetting of old classes, that we can fix with regularization strategies.
  arXiv  Detail & Related papers  (2020-11-30T20:45:56Z)
• Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation [128.03739769844736]
  Two neural co-attentions are incorporated into the classifier to capture cross-image semantic similarities and differences. In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference. Our algorithm sets new state-of-the-arts on all these settings, demonstrating well its efficacy and generalizability.
  arXiv  Detail & Related papers  (2020-07-03T21:53:46Z)
• Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels [35.586031601299034]
  We present a technique implementing class-incremental learning for semantic segmentation without using the labeled data the model was initially trained on. We show how to overcome these problems with a novel class-incremental learning technique, which nonetheless requires labels only for the new classes. We evaluate our method on the Cityscapes dataset, where we exceed the mIoU performance of all baselines by 3.5% absolute reaching a result.
  arXiv  Detail & Related papers  (2020-05-12T21:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.