Semi-supervised Semantic Segmentation with Prototype-based Consistency
Regularization
- URL: http://arxiv.org/abs/2210.04388v1
- Date: Mon, 10 Oct 2022 01:38:01 GMT
- Title: Semi-supervised Semantic Segmentation with Prototype-based Consistency
Regularization
- Authors: Hai-Ming Xu, Lingqiao Liu, Qiuchen Bian, Zhen Yang
- Abstract summary: Semi-supervised semantic segmentation requires the model to propagate the label information from limited annotated images to unlabeled ones.
A challenge for such a per-pixel prediction task is the large intra-class variation.
We propose a novel approach to regularize the distribution of within-class features to ease label propagation difficulty.
- Score: 20.4183741427867
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Semi-supervised semantic segmentation requires the model to effectively
propagate the label information from limited annotated images to unlabeled
ones. A challenge for such a per-pixel prediction task is the large intra-class
variation, i.e., regions belonging to the same class may exhibit a very
different appearance even in the same picture. This diversity will make the
label propagation hard from pixels to pixels. To address this problem, we
propose a novel approach to regularize the distribution of within-class
features to ease label propagation difficulty. Specifically, our approach
encourages the consistency between the prediction from a linear predictor and
the output from a prototype-based predictor, which implicitly encourages
features from the same pseudo-class to be close to at least one within-class
prototype while staying far from the other between-class prototypes. By further
incorporating CutMix operations and a carefully-designed prototype maintenance
strategy, we create a semi-supervised semantic segmentation algorithm that
demonstrates superior performance over the state-of-the-art methods from
extensive experimental evaluation on both Pascal VOC and Cityscapes benchmarks.
Related papers
- Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation [56.1776710527814]
Weakly Incremental Learning for Semantic (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-level labels.
A prevailing way to solve WILSS is the generation of seed areas for each new class, serving as a form of pixel-level supervision.
We propose an innovative, tendency-driven relationship of mutual exclusivity, meticulously tailored to govern the behavior of the seed areas.
arXiv Detail & Related papers (2024-04-18T08:23:24Z) - Decoupled Prototype Learning for Reliable Test-Time Adaptation [50.779896759106784]
Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference.
One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels.
This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise.
We propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation.
arXiv Detail & Related papers (2024-01-15T03:33:39Z) - Boundary-Refined Prototype Generation: A General End-to-End Paradigm for
Semi-Supervised Semantic Segmentation [34.88132191766558]
Prototype-based classification is a classical method in machine learning, and recently it has achieved remarkable success in semi-supervised semantic segmentation.
We propose a novel boundary-refined prototype generation (BRPG) method, which is incorporated into the whole training framework.
Our approach samples and clusters high- and low-confidence features separately based on a confidence threshold, aiming to generate prototypes closer to the class boundaries.
arXiv Detail & Related papers (2023-07-19T16:12:37Z) - ProtoSeg: Interpretable Semantic Segmentation with Prototypical Parts [12.959270094693254]
We introduce ProtoSeg, a novel model for interpretable semantic image segmentation.
To achieve accuracy comparable to baseline methods, we adapt the mechanism of prototypical parts.
We show that ProtoSeg discovers semantic concepts, in contrast to standard segmentation models.
arXiv Detail & Related papers (2023-01-28T19:14:32Z) - Rethinking Clustering-Based Pseudo-Labeling for Unsupervised
Meta-Learning [146.11600461034746]
Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling.
This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data.
We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
arXiv Detail & Related papers (2022-09-27T19:04:36Z) - Beyond the Prototype: Divide-and-conquer Proxies for Few-shot
Segmentation [63.910211095033596]
Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples.
We propose a simple yet versatile framework in the spirit of divide-and-conquer.
Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information.
arXiv Detail & Related papers (2022-04-21T06:21:14Z) - Diversified Multi-prototype Representation for Semi-supervised
Segmentation [9.994508738317585]
This work considers semi-supervised segmentation as a dense prediction problem based on prototype vector correlation.
Two regularization strategies are applied to ensure that all prototype vectors are considered by the network.
Experimental results on two benchmark medical segmentation datasets reveal our method's effectiveness in improving segmentation performance when few annotated images are available.
arXiv Detail & Related papers (2021-11-16T17:33:58Z) - Dual Prototypical Contrastive Learning for Few-shot Semantic
Segmentation [55.339405417090084]
We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task.
The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space.
We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
arXiv Detail & Related papers (2021-11-09T08:14:50Z) - Attentional Prototype Inference for Few-Shot Segmentation [128.45753577331422]
We propose attentional prototype inference (API), a probabilistic latent variable framework for few-shot segmentation.
We define a global latent variable to represent the prototype of each object category, which we model as a probabilistic distribution.
We conduct extensive experiments on four benchmarks, where our proposal obtains at least competitive and often better performance than state-of-the-art prototype-based methods.
arXiv Detail & Related papers (2021-05-14T06:58:44Z) - Semi-Supervised Semantic Segmentation with Cross-Consistency Training [8.894935073145252]
We present a novel cross-consistency based semi-supervised approach for semantic segmentation.
Our method achieves state-of-the-art results in several datasets.
arXiv Detail & Related papers (2020-03-19T20:10:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.