Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast
- URL: http://arxiv.org/abs/2110.07110v1
- Date: Thu, 14 Oct 2021 01:44:57 GMT
- Title: Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast
- Authors: Ye Du, Zehua Fu, Qingjie Liu, Yunhong Wang
- Abstract summary: Cross-view feature semantic consistency and intra(inter)-class compactness(dispersion) are explored.
We propose two novel pixel-to-prototype contrast regularization terms that are conducted cross different views and within per single view of an image.
Our method can be seamlessly incorporated into existing WSSS models without any changes to the base network.
- Score: 43.40192909920495
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Though image-level weakly supervised semantic segmentation (WSSS) has
achieved great progress with Class Activation Map (CAM) as the cornerstone, the
large supervision gap between classification and segmentation still hampers the
model to generate more complete and precise pseudo masks for segmentation.
In this study, we explore two implicit but intuitive constraints, i.e.,
cross-view feature semantic consistency and intra(inter)-class
compactness(dispersion), to narrow the supervision gap.
To this end, we propose two novel pixel-to-prototype contrast regularization
terms that are conducted cross different views and within per single view of an
image, respectively. Besides, we adopt two sample mining strategies, named
semi-hard prototype mining and hard pixel sampling, to better leverage hard
examples while reducing incorrect contrasts caused due to the absence of
precise pixel-wise labels.
Our method can be seamlessly incorporated into existing WSSS models without
any changes to the base network and does not incur any extra inference burden.
Experiments on standard benchmark show that our method consistently improves
two strong baselines by large margins, demonstrating the effectiveness of our
method. Specifically, built on top of SEAM, we improve the initial seed mIoU on
PASCAL VOC 2012 from 55.4% to 61.5%. Moreover, armed with our method, we
increase the segmentation mIoU of EPS from 70.8% to 73.6%, achieving new
state-of-the-art.
Related papers
- Fine-grained Background Representation for Weakly Supervised Semantic Segmentation [35.346567242839065]
This paper proposes a simple fine-grained background representation (FBR) method to discover and represent diverse BG semantics.
We present an active sampling strategy to mine the FG negatives on-the-fly, enabling efficient pixel-to-pixel intra-foreground contrastive learning.
Our method achieves 73.2 mIoU and 45.6 mIoU segmentation results on Pascal Voc and MS COCO test sets, respectively.
arXiv Detail & Related papers (2024-06-22T06:45:25Z) - RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised
Medical Image Segmentation [26.933651788004475]
We propose a novel semi-supervised segmentation method named Rectified Contrastive Pseudo Supervision (RCPS)
RCPS combines a rectified pseudo supervision and voxel-level contrastive learning to improve the effectiveness of semi-supervised segmentation.
Experimental results reveal that the proposed method yields better segmentation performance compared with the state-of-the-art methods in semi-supervised medical image segmentation.
arXiv Detail & Related papers (2023-01-13T12:03:58Z) - Rethinking Clustering-Based Pseudo-Labeling for Unsupervised
Meta-Learning [146.11600461034746]
Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling.
This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data.
We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
arXiv Detail & Related papers (2022-09-27T19:04:36Z) - Exploring Feature Representation Learning for Semi-supervised Medical
Image Segmentation [30.608293915653558]
We present a two-stage framework for semi-supervised medical image segmentation.
Key insight is to explore the feature representation learning with labeled and unlabeled (i.e., pseudo labeled) images.
A stage-adaptive contrastive learning method is proposed, containing a boundary-aware contrastive loss.
We present an aleatoric uncertainty-aware method, namely AUA, to generate higher-quality pseudo labels.
arXiv Detail & Related papers (2021-11-22T05:06:12Z) - Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation [22.920856071095915]
We present a novel semi-supervised semantic segmentation method which jointly achieves two desiderata of segmentation model regularities.
We leverage the pixel-level L2 loss and the pixel contrastive loss for the two purposes respectively.
arXiv Detail & Related papers (2021-08-20T07:04:33Z) - Semi-supervised Semantic Segmentation with Directional Context-aware
Consistency [66.49995436833667]
We focus on the semi-supervised segmentation problem where only a small set of labeled data is provided with a much larger collection of totally unlabeled images.
A preferred high-level representation should capture the contextual information while not losing self-awareness.
We present the Directional Contrastive Loss (DC Loss) to accomplish the consistency in a pixel-to-pixel manner.
arXiv Detail & Related papers (2021-06-27T03:42:40Z) - BoundarySqueeze: Image Segmentation as Boundary Squeezing [104.43159799559464]
We propose a novel method for fine-grained high-quality image segmentation of both objects and scenes.
Inspired by dilation and erosion from morphological image processing techniques, we treat the pixel level segmentation problems as squeezing object boundary.
Our method yields large gains on COCO, Cityscapes, for both instance and semantic segmentation and outperforms previous state-of-the-art PointRend in both accuracy and speed under the same setting.
arXiv Detail & Related papers (2021-05-25T04:58:51Z) - Railroad is not a Train: Saliency as Pseudo-pixel Supervision for Weakly
Supervised Semantic Segmentation [16.560870740946275]
Explicit Pseudo-pixel Supervision (EPS) learns from pixel-level feedback by combining two weak supervisions.
We devise a joint training strategy to fully utilize the complementary relationship between both information.
Our method can obtain accurate object boundaries and discard co-occurring pixels, thereby significantly improving the quality of pseudo-masks.
arXiv Detail & Related papers (2021-05-19T07:31:11Z) - Dense Contrastive Learning for Self-Supervised Visual Pre-Training [102.15325936477362]
We present dense contrastive learning, which implements self-supervised learning by optimizing a pairwise contrastive (dis)similarity loss at the pixel level between two views of input images.
Compared to the baseline method MoCo-v2, our method introduces negligible computation overhead (only 1% slower)
arXiv Detail & Related papers (2020-11-18T08:42:32Z) - Causal Intervention for Weakly-Supervised Semantic Segmentation [122.1846968696862]
We aim to generate better pixel-level pseudo-masks by using only image-level labels.
We propose a structural causal model to analyze the causalities among images, contexts, and class labels.
Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification.
arXiv Detail & Related papers (2020-09-26T09:26:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.