Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2203.00962v1
• Date: Wed, 2 Mar 2022 09:14:58 GMT
• Title: Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation
• Authors: Zhaozheng Chen, Tan Wang, Xiongwei Wu, Xian-Sheng Hua, Hanwang Zhang, Qianru Sun
• Abstract summary: Class activation maps (CAM) is arguably the most standard step of generating pseudo masks for semantic segmentation. Yet, the crux of the unsatisfactory pseudo masks is the binary cross-entropy loss (BCE) widely used in CAM. We introduce an embarrassingly simple yet surprisingly effective method: Reactivating the converged CAM with BCE by using softmax cross-entropy loss (SCE) The evaluation on both PASCAL VOC and MSCOCO shows that ReCAM not only generates high-quality masks, but also supports plug-and-play in any CAM variant with little overhead.
• Score: 88.55040177178442
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extracting class activation maps (CAM) is arguably the most standard step of generating pseudo masks for weakly-supervised semantic segmentation (WSSS). Yet, we find that the crux of the unsatisfactory pseudo masks is the binary cross-entropy loss (BCE) widely used in CAM. Specifically, due to the sum-over-class pooling nature of BCE, each pixel in CAM may be responsive to multiple classes co-occurring in the same receptive field. As a result, given a class, its hot CAM pixels may wrongly invade the area belonging to other classes, or the non-hot ones may be actually a part of the class. To this end, we introduce an embarrassingly simple yet surprisingly effective method: Reactivating the converged CAM with BCE by using softmax cross-entropy loss (SCE), dubbed \textbf{ReCAM}. Given an image, we use CAM to extract the feature pixels of each single class, and use them with the class label to learn another fully-connected layer (after the backbone) with SCE. Once converged, we extract ReCAM in the same way as in CAM. Thanks to the contrastive nature of SCE, the pixel response is disentangled into different classes and hence less mask ambiguity is expected. The evaluation on both PASCAL VOC and MS~COCO shows that ReCAM not only generates high-quality masks, but also supports plug-and-play in any CAM variant with little overhead.

Related papers

• BroadCAM: Outcome-agnostic Class Activation Mapping for Small-scale Weakly Supervised Applications [69.22739434619531]
  We propose an outcome-agnostic CAM approach, called BroadCAM, for small-scale weakly supervised applications. By evaluating BroadCAM on VOC2012 and BCSS-WSSS for WSSS and OpenImages30k for WSOL, BroadCAM demonstrates superior performance.
  arXiv  Detail & Related papers  (2023-09-07T06:45:43Z)
• Class-Incremental Exemplar Compression for Class-Incremental Learning [90.93462714376078]
  We propose an adaptive mask generation model called class-incremental masking (CIM) We conduct experiments on high-resolution CIL benchmarks including Food-101, ImageNet-100, and ImageNet-1000. We show that using the compressed exemplars by CIM can achieve a new state-of-the-art CIL accuracy, e.g., 4.8 percentage points higher than FOSTER on 10-Phase ImageNet-1000.
  arXiv  Detail & Related papers  (2023-03-24T14:51:20Z)
• Extracting Class Activation Maps from Non-Discriminative Features as well [23.968856513180032]
  Class activation maps (CAM) from a classification model often results in poor coverage on foreground objects. We introduce a new computation method for CAM that explicitly captures non-discriminative features as well. We call the resultant K cluster centers local prototypes - represent local semantics like the "head", "leg", and "body" of "sheep"
  arXiv  Detail & Related papers  (2023-03-18T04:47:42Z)
• Exploit CAM by itself: Complementary Learning System for Weakly Supervised Semantic Segmentation [59.24824050194334]
  This paper turns to an interesting working mechanism in agent learning named Complementary Learning System ( CLS) Motivated by this simple but effective learning pattern, we propose a General-Specific Learning Mechanism (GSLM) GSLM develops a General Learning Module (GLM) and a Specific Learning Module (SLM)
  arXiv  Detail & Related papers  (2023-03-04T16:16:47Z)
• Attention-based Class Activation Diffusion for Weakly-Supervised Semantic Segmentation [98.306533433627]
  extracting class activation maps (CAM) is a key step for weakly-supervised semantic segmentation (WSSS) This paper proposes a new method to couple CAM and Attention matrix in a probabilistic Diffusion way, and dub it AD-CAM. Experiments show that AD-CAM as pseudo labels can yield stronger WSSS models than the state-of-the-art variants of CAM.
  arXiv  Detail & Related papers  (2022-11-20T10:06:32Z)
• Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation [72.33139350241044]
  Weakly Supervised Semantic COCO (WSSS) based on image-level labels has attracted much attention due to low annotation costs. We propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Our SIPE achieves new state-of-the-art performance using only image-level labels.
  arXiv  Detail & Related papers  (2022-03-06T09:01:03Z)
• F-CAM: Full Resolution CAM via Guided Parametric Upscaling [20.609010268320013]
  Class Activation Mapping (CAM) methods have recently gained much attention for weakly-supervised object localization (WSOL) tasks. CAM methods are typically integrated within off-the-shelf CNN backbones, such as ResNet50. We introduce a generic method for parametric upscaling of CAMs that allows constructing accurate full resolution CAMs.
  arXiv  Detail & Related papers  (2021-09-15T04:45:20Z)
• Towards Learning Spatially Discriminative Feature Representations [26.554140976236052]
  We propose a novel loss function, termed as CAM-loss, to constrain the embedded feature maps with the class activation maps (CAMs) CAM-loss drives the backbone to express the features of target category and suppress the features of non-target categories or background. Experimental results show that CAM-loss is applicable to a variety of network structures and can be combined with mainstream regularization methods to improve the performance of image classification.
  arXiv  Detail & Related papers  (2021-09-03T08:04:17Z)
• Complementary Patch for Weakly Supervised Semantic Segmentation [25.990386258122726]
  We formulate the expansion of object regions in Class Activation Map (CAM) as an increase in information. We propose a novel Complementary Patch (CP) Representation and prove that the information of the sum of the CAMs by a pair of input images with complementary hidden (patched) parts, namely CP Pair, is greater than or equal to the information of the baseline CAM. To further improve the quality of the CAMs, we propose a Pixel-Region Correlation Module (PRCM) to augment the contextual information.
  arXiv  Detail & Related papers  (2021-08-09T07:50:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.