MuSCLe: A Multi-Strategy Contrastive Learning Framework for Weakly Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2201.07021v1
• Date: Tue, 18 Jan 2022 14:38:50 GMT
• Title: MuSCLe: A Multi-Strategy Contrastive Learning Framework for Weakly Supervised Semantic Segmentation
• Authors: Kunhao Yuan, Gerald Schaefer, Yu-Kun Lai, Yifan Wang, Xiyao Liu, Lin Guan, Hui Fang
• Abstract summary: Weakly supervised semantic segmentation (WSSS) relies on weak labels such as image level annotations rather than pixel level annotations required by supervised semantic segmentation (SSS) methods. We propose a novel Multi-Strategy Contrastive Learning (MuSCLe) framework to obtain enhanced feature representations and improve WSSS performance.
• Score: 39.858844102571176
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weakly supervised semantic segmentation (WSSS) has gained significant popularity since it relies only on weak labels such as image level annotations rather than pixel level annotations required by supervised semantic segmentation (SSS) methods. Despite drastically reduced annotation costs, typical feature representations learned from WSSS are only representative of some salient parts of objects and less reliable compared to SSS due to the weak guidance during training. In this paper, we propose a novel Multi-Strategy Contrastive Learning (MuSCLe) framework to obtain enhanced feature representations and improve WSSS performance by exploiting similarity and dissimilarity of contrastive sample pairs at image, region, pixel and object boundary levels. Extensive experiments demonstrate the effectiveness of our method and show that MuSCLe outperforms the current state-of-the-art on the widely used PASCAL VOC 2012 dataset.

Related papers

• Weakly-supervised Semantic Segmentation via Dual-stream Contrastive Learning of Cross-image Contextual Information [10.77139542242678]
  Weakly supervised semantic segmentation (WSSS) aims at learning a semantic segmentation model with only image-level tags. Most current WSSS methods focus on a limited single image (pixel-wise) information while ignoring the valuable inter-image (semantic-wise) information.
  arXiv  Detail & Related papers  (2024-05-08T09:35:26Z)
• SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentation [53.010417880335424]
  Semi-supervised temporal action segmentation (SS-TA) aims to perform frame-wise classification in long untrimmed videos. Recent studies have shown the potential of contrastive learning in unsupervised representation learning using unlabelled data. We propose a novel Semantic-guided Multi-level Contrast scheme with a Neighbourhood-Consistency-Aware unit (SMC-NCA) to extract strong frame-wise representations.
  arXiv  Detail & Related papers  (2023-12-19T17:26:44Z)
• Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation [119.009033745244]
  This paper presents a Self-supervised Low-Rank Network ( SLRNet) for single-stage weakly supervised semantic segmentation (WSSS) and semi-supervised semantic segmentation (SSSS) SLRNet uses cross-view self-supervision, that is, it simultaneously predicts several attentive LR representations from different views of an image to learn precise pseudo-labels. Experiments on the Pascal VOC 2012, COCO, and L2ID datasets demonstrate that our SLRNet outperforms both state-of-the-art WSSS and SSSS methods with a variety of different settings.
  arXiv  Detail & Related papers  (2022-03-19T09:19:55Z)
• Novel Class Discovery in Semantic Segmentation [104.30729847367104]
  We introduce a new setting of Novel Class Discovery in Semantic (NCDSS) It aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image. We propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels.
  arXiv  Detail & Related papers  (2021-12-03T13:31:59Z)
• Dense Contrastive Visual-Linguistic Pretraining [53.61233531733243]
  Several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. We propose unbiased Dense Contrastive Visual-Linguistic Pretraining to replace the region regression and classification with cross-modality region contrastive learning.
  arXiv  Detail & Related papers  (2021-09-24T07:20:13Z)
• Semi-supervised Semantic Segmentation with Directional Context-aware Consistency [66.49995436833667]
  We focus on the semi-supervised segmentation problem where only a small set of labeled data is provided with a much larger collection of totally unlabeled images. A preferred high-level representation should capture the contextual information while not losing self-awareness. We present the Directional Contrastive Loss (DC Loss) to accomplish the consistency in a pixel-to-pixel manner.
  arXiv  Detail & Related papers  (2021-06-27T03:42:40Z)
• Towards Single Stage Weakly Supervised Semantic Segmentation [2.28438857884398]
  We present a single-stage approach to weakly supervised semantic segmentation. We use point annotations to generate reliable, on-the-fly pseudo-masks. We significantly outperform other SOTA WSSS methods on recent real-world datasets.
  arXiv  Detail & Related papers  (2021-06-18T18:34:50Z)
• Few-Shot Semantic Segmentation Augmented with Image-Level Weak Annotations [23.02986307143718]
  Recent progress in fewshot semantic segmentation tackles the issue by only a few pixel-level annotated examples. Our key idea is to learn a better prototype representation of the class by fusing the knowledge from the image-level labeled data. We propose a new framework, called PAIA, to learn the class prototype representation in a metric space by integrating image-level annotations.
  arXiv  Detail & Related papers  (2020-07-03T04:58:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.