Complementary Patch for Weakly Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2108.03852v1
• Date: Mon, 9 Aug 2021 07:50:54 GMT
• Title: Complementary Patch for Weakly Supervised Semantic Segmentation
• Authors: Fei Zhang, Chaochen Gu, Chenyue Zhang, Yuchao Dai
• Abstract summary: We formulate the expansion of object regions in Class Activation Map (CAM) as an increase in information. We propose a novel Complementary Patch (CP) Representation and prove that the information of the sum of the CAMs by a pair of input images with complementary hidden (patched) parts, namely CP Pair, is greater than or equal to the information of the baseline CAM. To further improve the quality of the CAMs, we propose a Pixel-Region Correlation Module (PRCM) to augment the contextual information.
• Score: 25.990386258122726
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weakly Supervised Semantic Segmentation (WSSS) based on image-level labels has been greatly advanced by exploiting the outputs of Class Activation Map (CAM) to generate the pseudo labels for semantic segmentation. However, CAM merely discovers seeds from a small number of regions, which may be insufficient to serve as pseudo masks for semantic segmentation. In this paper, we formulate the expansion of object regions in CAM as an increase in information. From the perspective of information theory, we propose a novel Complementary Patch (CP) Representation and prove that the information of the sum of the CAMs by a pair of input images with complementary hidden (patched) parts, namely CP Pair, is greater than or equal to the information of the baseline CAM. Therefore, a CAM with more information related to object seeds can be obtained by narrowing down the gap between the sum of CAMs generated by the CP Pair and the original CAM. We propose a CP Network (CPN) implemented by a triplet network and three regularization functions. To further improve the quality of the CAMs, we propose a Pixel-Region Correlation Module (PRCM) to augment the contextual information by using object-region relations between the feature maps and the CAMs. Experimental results on the PASCAL VOC 2012 datasets show that our proposed method achieves a new state-of-the-art in WSSS, validating the effectiveness of our CP Representation and CPN.

Related papers

• Beyond Discriminative Regions: Saliency Maps as Alternatives to CAMs for Weakly Supervised Semantic Segmentation [7.518769185369959]
  This paper provides a comprehensive comparison between saliencies and CAMs for Weakly Supervised Semantic (WS3) We demonstrate the effectiveness of saliencies in addressing the limitation of CAMs through our empirical studies on benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-21T21:30:48Z)
• Exploit CAM by itself: Complementary Learning System for Weakly Supervised Semantic Segmentation [59.24824050194334]
  This paper turns to an interesting working mechanism in agent learning named Complementary Learning System ( CLS) Motivated by this simple but effective learning pattern, we propose a General-Specific Learning Mechanism (GSLM) GSLM develops a General Learning Module (GLM) and a Specific Learning Module (SLM)
  arXiv  Detail & Related papers  (2023-03-04T16:16:47Z)
• Attention-based Class Activation Diffusion for Weakly-Supervised Semantic Segmentation [98.306533433627]
  extracting class activation maps (CAM) is a key step for weakly-supervised semantic segmentation (WSSS) This paper proposes a new method to couple CAM and Attention matrix in a probabilistic Diffusion way, and dub it AD-CAM. Experiments show that AD-CAM as pseudo labels can yield stronger WSSS models than the state-of-the-art variants of CAM.
  arXiv  Detail & Related papers  (2022-11-20T10:06:32Z)
• Saliency Guided Inter- and Intra-Class Relation Constraints for Weakly Supervised Semantic Segmentation [66.87777732230884]
  We propose a saliency guided Inter- and Intra-Class Relation Constrained (I$2$CRC) framework to assist the expansion of the activated object regions. We also introduce an object guided label refinement module to take a full use of both the segmentation prediction and the initial labels for obtaining superior pseudo-labels.
  arXiv  Detail & Related papers  (2022-06-20T03:40:56Z)
• Importance Sampling CAMs for Weakly-Supervised Segmentation [16.86352815414646]
  Class activation maps (CAMs) can be used to localize and segment objects in images by means of class activation maps (CAMs) In this work, we approach both problems with two contributions for improving CAM learning. We conduct experiments on the PASCAL VOC 2012 benchmark dataset to demonstrate that these modifications significantly increase the performance in terms of contour accuracy.
  arXiv  Detail & Related papers  (2022-03-23T14:54:29Z)
• TransCAM: Transformer Attention-based CAM Refinement for Weakly Supervised Semantic Segmentation [19.333543299407832]
  We propose TransCAM, a Conformer-based solution to weakly supervised semantic segmentation. We show that TransCAM achieves a new state-of-the-art performance of 69.3% and 69.6% on the respective PASCAL VOC 2012 validation and test sets.
  arXiv  Detail & Related papers  (2022-03-14T16:17:18Z)
• Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation [72.33139350241044]
  Weakly Supervised Semantic COCO (WSSS) based on image-level labels has attracted much attention due to low annotation costs. We propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Our SIPE achieves new state-of-the-art performance using only image-level labels.
  arXiv  Detail & Related papers  (2022-03-06T09:01:03Z)
• PCAM: Product of Cross-Attention Matrices for Rigid Registration of Point Clouds [79.99653758293277]
  PCAM is a neural network whose key element is a pointwise product of cross-attention matrices. We show that PCAM achieves state-of-the-art results among methods which, like us, solve steps (a) and (b) jointly via deepnets.
  arXiv  Detail & Related papers  (2021-10-04T09:23:27Z)
• Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
  TRansformer-based Few-shot Semantic segmentation method (TRFS) Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
  arXiv  Detail & Related papers  (2021-08-04T20:09:21Z)
• Puzzle-CAM: Improved localization via matching partial and full features [0.5482532589225552]
  Weakly-supervised semantic segmentation (WSSS) is introduced to narrow the gap for semantic segmentation performance from pixel-level supervision to image-level supervision. Most advanced approaches are based on class activation maps (CAMs) to generate pseudo-labels to train the segmentation network. We propose Puzzle-CAM, a process that minimizes differences between the features from separate patches and the whole image. In experiments, Puzzle-CAM outperformed previous state-of-the-art methods using the same labels for supervision on the PASCAL VOC 2012 dataset.
  arXiv  Detail & Related papers  (2021-01-27T08:19:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.