Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised
Semantic Segmentation with Its Class Label
- URL: http://arxiv.org/abs/2402.17555v1
- Date: Tue, 27 Feb 2024 14:51:56 GMT
- Title: Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised
Semantic Segmentation with Its Class Label
- Authors: Xinliang Zhang, Lei Zhu, Hangzhou He, Lujia Jin, Yanye Lu
- Abstract summary: We propose a class-driven scribble promotion network, which utilizes both scribble annotations and pseudo-labels informed by image-level classes and global semantics for supervision.
Experiments on the ScribbleSup dataset with different qualities of scribble annotations outperform all the previous methods, demonstrating the superiority and robustness of our method.
- Score: 16.745019028033518
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Scribble-based weakly-supervised semantic segmentation using sparse scribble
supervision is gaining traction as it reduces annotation costs when compared to
fully annotated alternatives. Existing methods primarily generate pseudo-labels
by diffusing labeled pixels to unlabeled ones with local cues for supervision.
However, this diffusion process fails to exploit global semantics and
class-specific cues, which are important for semantic segmentation. In this
study, we propose a class-driven scribble promotion network, which utilizes
both scribble annotations and pseudo-labels informed by image-level classes and
global semantics for supervision. Directly adopting pseudo-labels might
misguide the segmentation model, thus we design a localization rectification
module to correct foreground representations in the feature space. To further
combine the advantages of both supervisions, we also introduce a distance
entropy loss for uncertainty reduction, which adapts per-pixel confidence
weights according to the reliable region determined by the scribble and
pseudo-label's boundary. Experiments on the ScribbleSup dataset with different
qualities of scribble annotations outperform all the previous methods,
demonstrating the superiority and robustness of our method.The code is
available at
https://github.com/Zxl19990529/Class-driven-Scribble-Promotion-Network.
Related papers
- Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets [51.74296438621836]
We introduce Scribbles for All, a label and training data generation algorithm for semantic segmentation trained on scribble labels.
The main limitation of scribbles as source for weak supervision is the lack of challenging datasets for scribble segmentation.
Scribbles for All provides scribble labels for several popular segmentation datasets and provides an algorithm to automatically generate scribble labels for any dataset with dense annotations.
arXiv Detail & Related papers (2024-08-22T15:29:08Z) - Semi-supervised Semantic Segmentation Meets Masked Modeling:Fine-grained
Locality Learning Matters in Consistency Regularization [31.333862320143968]
Semi-supervised semantic segmentation aims to utilize limited labeled images and abundant unlabeled images to achieve label-efficient learning.
We propose a novel framework called textttMaskMatch, which enables fine-grained locality learning to achieve better dense segmentation.
arXiv Detail & Related papers (2023-12-14T03:28:53Z) - Class Enhancement Losses with Pseudo Labels for Zero-shot Semantic
Segmentation [40.09476732999614]
Mask proposal models have significantly improved the performance of zero-shot semantic segmentation.
The use of a background' embedding during training in these methods is problematic as the resulting model tends to over-learn and assign all unseen classes as the background class instead of their correct labels.
This paper proposes novel class enhancement losses to bypass the use of the background embbedding during training, and simultaneously exploit the semantic relationship between text embeddings and mask proposals by ranking the similarity scores.
arXiv Detail & Related papers (2023-01-18T06:55:02Z) - Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud [69.36717778451667]
Existing methods for large-scale point cloud semantic segmentation require expensive, tedious and error-prone manual point-wise annotations.
We propose an effective weakly supervised method containing two components to solve the problem.
The experimental results show the large gain against existing weakly supervised and comparable results to fully supervised methods.
arXiv Detail & Related papers (2022-12-09T09:42:26Z) - LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds.
Our method co-designs an efficient labeling process with semi/weakly supervised learning.
Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
arXiv Detail & Related papers (2022-10-14T19:13:36Z) - Learning Semantic Correspondence with Sparse Annotations [66.37298464505261]
Finding dense semantic correspondence is a fundamental problem in computer vision.
We propose a teacher-student learning paradigm for generating dense pseudo-labels.
We also develop two novel strategies for denoising pseudo-labels.
arXiv Detail & Related papers (2022-08-15T02:24:18Z) - Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive
Learning [28.498782661888775]
We formulate weakly supervised segmentation as a semi-supervised metric learning problem.
We propose 4 types of contrastive relationships between pixels and segments in the feature space.
We deliver a universal weakly supervised segmenter with significant gains on Pascal VOC and DensePose.
arXiv Detail & Related papers (2021-05-03T15:49:01Z) - A Closer Look at Self-training for Zero-Label Semantic Segmentation [53.4488444382874]
Being able to segment unseen classes not observed during training is an important technical challenge in deep learning.
Prior zero-label semantic segmentation works approach this task by learning visual-semantic embeddings or generative models.
We propose a consistency regularizer to filter out noisy pseudo-labels by taking the intersections of the pseudo-labels generated from different augmentations of the same image.
arXiv Detail & Related papers (2021-04-21T14:34:33Z) - PCAMs: Weakly Supervised Semantic Segmentation Using Point Supervision [12.284208932393073]
This paper presents a novel procedure for producing semantic segmentation from images given some point level annotations.
We propose training a CNN that is normally fully supervised using our pseudo labels in place of ground truth labels.
Our method achieves state of the art results for point supervised semantic segmentation on the PASCAL VOC 2012 dataset citeeveringham2010pascal, even outperforming state of the art methods for stronger bounding box and squiggle supervision.
arXiv Detail & Related papers (2020-07-10T21:25:27Z) - Weakly-Supervised Salient Object Detection via Scribble Annotations [54.40518383782725]
We propose a weakly-supervised salient object detection model to learn saliency from scribble labels.
We present a new metric, termed saliency structure measure, to measure the structure alignment of the predicted saliency maps.
Our method not only outperforms existing weakly-supervised/unsupervised methods, but also is on par with several fully-supervised state-of-the-art models.
arXiv Detail & Related papers (2020-03-17T12:59:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.