Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization

• URL: http://arxiv.org/abs/2210.04388v1
• Date: Mon, 10 Oct 2022 01:38:01 GMT
• Title: Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization
• Authors: Hai-Ming Xu, Lingqiao Liu, Qiuchen Bian, Zhen Yang
• Abstract summary: Semi-supervised semantic segmentation requires the model to propagate the label information from limited annotated images to unlabeled ones. A challenge for such a per-pixel prediction task is the large intra-class variation. We propose a novel approach to regularize the distribution of within-class features to ease label propagation difficulty.
• Score: 20.4183741427867
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Semi-supervised semantic segmentation requires the model to effectively propagate the label information from limited annotated images to unlabeled ones. A challenge for such a per-pixel prediction task is the large intra-class variation, i.e., regions belonging to the same class may exhibit a very different appearance even in the same picture. This diversity will make the label propagation hard from pixels to pixels. To address this problem, we propose a novel approach to regularize the distribution of within-class features to ease label propagation difficulty. Specifically, our approach encourages the consistency between the prediction from a linear predictor and the output from a prototype-based predictor, which implicitly encourages features from the same pseudo-class to be close to at least one within-class prototype while staying far from the other between-class prototypes. By further incorporating CutMix operations and a carefully-designed prototype maintenance strategy, we create a semi-supervised semantic segmentation algorithm that demonstrates superior performance over the state-of-the-art methods from extensive experimental evaluation on both Pascal VOC and Cityscapes benchmarks.

Related papers

• Mind the Gap Between Prototypes and Images in Cross-domain Finetuning [64.97317635355124]
  We propose a contrastive prototype-image adaptation (CoPA) to adapt different transformations respectively for prototypes and images. Experiments on Meta-Dataset demonstrate that CoPA achieves the state-of-the-art performance more efficiently.
  arXiv  Detail & Related papers  (2024-10-16T11:42:11Z)
• Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation [56.1776710527814]
  Weakly Incremental Learning for Semantic (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-level labels. A prevailing way to solve WILSS is the generation of seed areas for each new class, serving as a form of pixel-level supervision. We propose an innovative, tendency-driven relationship of mutual exclusivity, meticulously tailored to govern the behavior of the seed areas.
  arXiv  Detail & Related papers  (2024-04-18T08:23:24Z)
• Boundary-Refined Prototype Generation: A General End-to-End Paradigm for Semi-Supervised Semantic Segmentation [23.00156170789867]
  Semi-supervised semantic segmentation has attracted increasing attention in computer vision. Current approaches isolate prototype generation from the main training framework. We propose a novel end-to-end boundary-refined prototype generation (BRPG) method.
  arXiv  Detail & Related papers  (2023-07-19T16:12:37Z)
• ProtoSeg: Interpretable Semantic Segmentation with Prototypical Parts [12.959270094693254]
  We introduce ProtoSeg, a novel model for interpretable semantic image segmentation. To achieve accuracy comparable to baseline methods, we adapt the mechanism of prototypical parts. We show that ProtoSeg discovers semantic concepts, in contrast to standard segmentation models.
  arXiv  Detail & Related papers  (2023-01-28T19:14:32Z)
• Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning [146.11600461034746]
  Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
  arXiv  Detail & Related papers  (2022-09-27T19:04:36Z)
• Beyond the Prototype: Divide-and-conquer Proxies for Few-shot Segmentation [63.910211095033596]
  Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples. We propose a simple yet versatile framework in the spirit of divide-and-conquer. Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information.
  arXiv  Detail & Related papers  (2022-04-21T06:21:14Z)
• Diversified Multi-prototype Representation for Semi-supervised Segmentation [9.994508738317585]
  This work considers semi-supervised segmentation as a dense prediction problem based on prototype vector correlation. Two regularization strategies are applied to ensure that all prototype vectors are considered by the network. Experimental results on two benchmark medical segmentation datasets reveal our method's effectiveness in improving segmentation performance when few annotated images are available.
  arXiv  Detail & Related papers  (2021-11-16T17:33:58Z)
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation [55.339405417090084]
  We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task. The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space. We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
  arXiv  Detail & Related papers  (2021-11-09T08:14:50Z)
• Attentional Prototype Inference for Few-Shot Segmentation [128.45753577331422]
  We propose attentional prototype inference (API), a probabilistic latent variable framework for few-shot segmentation. We define a global latent variable to represent the prototype of each object category, which we model as a probabilistic distribution. We conduct extensive experiments on four benchmarks, where our proposal obtains at least competitive and often better performance than state-of-the-art prototype-based methods.
  arXiv  Detail & Related papers  (2021-05-14T06:58:44Z)
• Semi-Supervised Semantic Segmentation with Cross-Consistency Training [8.894935073145252]
  We present a novel cross-consistency based semi-supervised approach for semantic segmentation. Our method achieves state-of-the-art results in several datasets.
  arXiv  Detail & Related papers  (2020-03-19T20:10:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.