Weakly-Supervised Semantic Segmentation by Learning Label Uncertainty

• URL: http://arxiv.org/abs/2110.05926v1
• Date: Tue, 12 Oct 2021 12:19:22 GMT
• Title: Weakly-Supervised Semantic Segmentation by Learning Label Uncertainty
• Authors: Robby Neven, Davy Neven, Bert De Brabandere, Marc Proesmans and Toon Goedem\'e
• Abstract summary: We present a new loss function to train a segmentation network with only a small subset of pixel-perfect labels. Our loss trains the network to learn a label uncertainty within the bounding-box, which can be leveraged to perform online bootstrapping. We trained each task on a dataset comprised of only 18% pixel-perfect and 82% bounding-box labels.
• Score: 8.074019565026544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Since the rise of deep learning, many computer vision tasks have seen significant advancements. However, the downside of deep learning is that it is very data-hungry. Especially for segmentation problems, training a deep neural net requires dense supervision in the form of pixel-perfect image labels, which are very costly. In this paper, we present a new loss function to train a segmentation network with only a small subset of pixel-perfect labels, but take the advantage of weakly-annotated training samples in the form of cheap bounding-box labels. Unlike recent works which make use of box-to-mask proposal generators, our loss trains the network to learn a label uncertainty within the bounding-box, which can be leveraged to perform online bootstrapping (i.e. transforming the boxes to segmentation masks), while training the network. We evaluated our method on binary segmentation tasks, as well as a multi-class segmentation task (CityScapes vehicles and persons). We trained each task on a dataset comprised of only 18% pixel-perfect and 82% bounding-box labels, and compared the results to a baseline model trained on a completely pixel-perfect dataset. For the binary segmentation tasks, our method achieves an IoU score which is ~98.33% as good as our baseline model, while for the multi-class task, our method is 97.12% as good as our baseline model (77.5 vs. 79.8 mIoU).

Related papers

• You Only Need One Thing One Click: Self-Training for Weakly Supervised 3D Scene Understanding [107.06117227661204]
  We propose One Thing One Click'', meaning that the annotator only needs to label one point per object. We iteratively conduct the training and label propagation, facilitated by a graph propagation module. Our model can be compatible to 3D instance segmentation equipped with a point-clustering strategy.
  arXiv  Detail & Related papers  (2023-03-26T13:57:00Z)
• Semantic Segmentation with Active Semi-Supervised Learning [23.79742108127707]
  We propose a novel algorithm that combines active learning and semi-supervised learning. Our method obtains over 95% of the network's performance on the full-training set.
  arXiv  Detail & Related papers  (2022-03-21T04:16:25Z)
• Mars Terrain Segmentation with Less Labels [1.1745324895296465]
  This research proposes a semi-supervised learning framework for Mars terrain segmentation. It incorporates a backbone module which is trained using a contrastive loss function and an output atrous convolution module. The proposed model is able to achieve a segmentation accuracy of 91.1% using only 161 training images.
  arXiv  Detail & Related papers  (2022-02-01T22:25:15Z)
• Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images [0.2538209532048867]
  We propose a semi-weakly supervised contrastive learning framework for representation learning using semi-weakly annotated images. We empirically validate the transfer learning performance of SWCL on seven public retinal fundus datasets.
  arXiv  Detail & Related papers  (2021-08-04T15:50:09Z)
• A Closer Look at Self-training for Zero-Label Semantic Segmentation [53.4488444382874]
  Being able to segment unseen classes not observed during training is an important technical challenge in deep learning. Prior zero-label semantic segmentation works approach this task by learning visual-semantic embeddings or generative models. We propose a consistency regularizer to filter out noisy pseudo-labels by taking the intersections of the pseudo-labels generated from different augmentations of the same image.
  arXiv  Detail & Related papers  (2021-04-21T14:34:33Z)
• Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization [112.68171734288237]
  We propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels. We learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of unlabeled images. We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization.
  arXiv  Detail & Related papers  (2021-04-12T21:41:25Z)
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation [78.36781565047656]
  We propose "One Thing One Click," meaning that the annotator only needs to label one point per object. We iteratively conduct the training and label propagation, facilitated by a graph propagation module. Our results are also comparable to those of the fully supervised counterparts.
  arXiv  Detail & Related papers  (2021-04-06T02:27:25Z)
• Contrastive Learning for Label-Efficient Semantic Segmentation [44.10416030868873]
  Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled data. Deep CNNs trained with the de facto cross-entropy loss can easily overfit to small amounts of labeled data. We propose a simple and effective contrastive learning-based training strategy in which we first pretrain the network using a pixel-wise, label-based contrastive loss.
  arXiv  Detail & Related papers  (2020-12-13T07:05:39Z)
• Big Self-Supervised Models are Strong Semi-Supervised Learners [116.00752519907725]
  We show that it is surprisingly effective for semi-supervised learning on ImageNet. A key ingredient of our approach is the use of big (deep and wide) networks during pretraining and fine-tuning. We find that, the fewer the labels, the more this approach (task-agnostic use of unlabeled data) benefits from a bigger network.
  arXiv  Detail & Related papers  (2020-06-17T17:48:22Z)
• Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning [86.45526827323954]
  Weakly-supervised semantic segmentation is a challenging task as no pixel-wise label information is provided for training. We propose an iterative algorithm to learn such pairwise relations. We show that the proposed algorithm performs favorably against the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-19T10:32:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.