Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2110.14309v1
• Date: Wed, 27 Oct 2021 09:43:40 GMT
• Title: Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentation
• Authors: Weixuan Sun, Jing Zhang, Nick Barnes
• Abstract summary: We propose a class-conditional inference strategy and an activation aware mask refinement loss function to generate better pseudo labels. Our method achieves superior WSSS results without requiring re-training of the classifier.
• Score: 27.269847900950943
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image-level weakly supervised semantic segmentation (WSSS) relies on class activation maps (CAMs) for pseudo labels generation. As CAMs only highlight the most discriminative regions of objects, the generated pseudo labels are usually unsatisfactory to serve directly as supervision. To solve this, most existing approaches follow a multi-training pipeline to refine CAMs for better pseudo-labels, which includes: 1) re-training the classification model to generate CAMs; 2) post-processing CAMs to obtain pseudo labels; and 3) training a semantic segmentation model with the obtained pseudo labels. However, this multi-training pipeline requires complicated adjustment and additional time. To address this, we propose a class-conditional inference strategy and an activation aware mask refinement loss function to generate better pseudo labels without re-training the classifier. The class conditional inference-time approach is presented to separately and iteratively reveal the classification network's hidden object activation to generate more complete response maps. Further, our activation aware mask refinement loss function introduces a novel way to exploit saliency maps during segmentation training and refine the foreground object masks without suppressing background objects. Our method achieves superior WSSS results without requiring re-training of the classifier.

Related papers

• Dynamic Pseudo Label Optimization in Point-Supervised Nuclei Segmentation [17.698430642075813]
  We propose a framework named DoNuSeg, enabling textbfDynamic pseudo label textbfOptimization in point-supervised textbfNuclei textbfSegmentation. DoNuSeg takes advantage of class activation maps (CAMs) to adaptively capture regions with semantics similar to annotated points.
  arXiv  Detail & Related papers  (2024-06-24T08:20:53Z)
• Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation [79.05949524349005]
  We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps. We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
  arXiv  Detail & Related papers  (2024-03-02T10:03:21Z)
• Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation [30.812323329239614]
  Weakly supervised semantic segmentation (WSSS) aims to bypass the need for laborious pixel-level annotation by using only image-level annotation. Most existing methods rely on Class Activation Maps (CAM) to derive pixel-level pseudo-labels. We introduce a simple yet effective method harnessing the Segment Anything Model (SAM), a class-agnostic foundation model capable of producing fine-grained instance masks of objects, parts, and subparts.
  arXiv  Detail & Related papers  (2023-05-09T23:24:09Z)
• Saliency Guided Inter- and Intra-Class Relation Constraints for Weakly Supervised Semantic Segmentation [66.87777732230884]
  We propose a saliency guided Inter- and Intra-Class Relation Constrained (I$2$CRC) framework to assist the expansion of the activated object regions. We also introduce an object guided label refinement module to take a full use of both the segmentation prediction and the initial labels for obtaining superior pseudo-labels.
  arXiv  Detail & Related papers  (2022-06-20T03:40:56Z)
• Threshold Matters in WSSS: Manipulating the Activation for the Robust and Accurate Segmentation Model Against Thresholds [16.6833745997519]
  Weakly-supervised semantic segmentation (WSSS) has recently gained much attention for its promise to train segmentation models only with image-level labels. Existing WSSS methods commonly argue that the sparse coverage of CAM incurs the performance bottleneck of WSSS. This paper provides analytical and empirical evidence that the actual bottleneck may not be sparse coverage but a global thresholding scheme applied after CAM.
  arXiv  Detail & Related papers  (2022-03-30T04:26:14Z)
• Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation [88.49669148290306]
  We propose a novel weakly supervised multi-task framework called AuxSegNet to leverage saliency detection and multi-label image classification as auxiliary tasks. Inspired by their similar structured semantics, we also propose to learn a cross-task global pixel-level affinity map from the saliency and segmentation representations. The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks.
  arXiv  Detail & Related papers  (2021-07-25T11:39:58Z)
• Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification [84.72303377833732]
  Unsupervised object re-identification targets at learning discriminative representations for object retrieval without any annotations. We propose to estimate pseudo label similarities between consecutive training generations with clustering consensus and refine pseudo labels with temporally propagated and ensembled pseudo labels. The proposed pseudo label refinery strategy is simple yet effective and can be seamlessly integrated into existing clustering-based unsupervised re-identification methods.
  arXiv  Detail & Related papers  (2021-06-11T02:42:42Z)
• A Closer Look at Self-training for Zero-Label Semantic Segmentation [53.4488444382874]
  Being able to segment unseen classes not observed during training is an important technical challenge in deep learning. Prior zero-label semantic segmentation works approach this task by learning visual-semantic embeddings or generative models. We propose a consistency regularizer to filter out noisy pseudo-labels by taking the intersections of the pseudo-labels generated from different augmentations of the same image.
  arXiv  Detail & Related papers  (2021-04-21T14:34:33Z)
• Zoom-CAM: Generating Fine-grained Pixel Annotations from Image Labels [15.664293530106637]
  Zoom-CAM captures fine-grained small-scale objects for various discriminative class instances. We focus on generating pixel-level pseudo-labels from class labels. For weakly supervised semantic segmentation our generated pseudo-labels improve a state of the art model by 1.1%.
  arXiv  Detail & Related papers  (2020-10-16T22:06:43Z)
• Weakly-Supervised Semantic Segmentation via Sub-category Exploration [73.03956876752868]
  We propose a simple yet effective approach to enforce the network to pay attention to other parts of an object. Specifically, we perform clustering on image features to generate pseudo sub-categories labels within each annotated parent class. We conduct extensive analysis to validate the proposed method and show that our approach performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-08-03T20:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.