Warp-Refine Propagation: Semi-Supervised Auto-labeling via Cycle-consistency

• URL: http://arxiv.org/abs/2109.13432v1
• Date: Tue, 28 Sep 2021 02:04:18 GMT
• Title: Warp-Refine Propagation: Semi-Supervised Auto-labeling via Cycle-consistency
• Authors: Aditya Ganeshan, Alexis Vallet, Yasunori Kudo, Shin-ichi Maeda, Tommi Kerola, Rares Ambrus, Dennis Park, Adrien Gaidon
• Abstract summary: We propose a novel label propagation method that combines semantic cues with geometric cues to efficiently auto-label videos. Our method learns to refine geometrically-warped labels and infuse them with learned semantic priors in a semi-supervised setting. We quantitatively show that our method improves label-propagation by a noteworthy margin of 13.1 mIoU on the ApolloScape dataset.
• Score: 27.77065474840873
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning models for semantic segmentation rely on expensive, large-scale, manually annotated datasets. Labelling is a tedious process that can take hours per image. Automatically annotating video sequences by propagating sparsely labeled frames through time is a more scalable alternative. In this work, we propose a novel label propagation method, termed Warp-Refine Propagation, that combines semantic cues with geometric cues to efficiently auto-label videos. Our method learns to refine geometrically-warped labels and infuse them with learned semantic priors in a semi-supervised setting by leveraging cycle consistency across time. We quantitatively show that our method improves label-propagation by a noteworthy margin of 13.1 mIoU on the ApolloScape dataset. Furthermore, by training with the auto-labelled frames, we achieve competitive results on three semantic-segmentation benchmarks, improving the state-of-the-art by a large margin of 1.8 and 3.61 mIoU on NYU-V2 and KITTI, while matching the current best results on Cityscapes.

Related papers

• TrajSSL: Trajectory-Enhanced Semi-Supervised 3D Object Detection [59.498894868956306]
  Pseudo-labeling approaches to semi-supervised learning adopt a teacher-student framework. We leverage pre-trained motion-forecasting models to generate object trajectories on pseudo-labeled data. Our approach improves pseudo-label quality in two distinct manners.
  arXiv  Detail & Related papers  (2024-09-17T05:35:00Z)
• Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment [62.73503467108322]
  This topic is widely studied in 3D point cloud segmentation due to the difficulty of annotating point clouds densely. Until recently, pseudo-labels have been widely employed to facilitate training with limited ground-truth labels. Existing pseudo-labeling approaches could suffer heavily from the noises and variations in unlabelled data. We propose a novel learning strategy to regularize the pseudo-labels generated for training, thus effectively narrowing the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2024-08-29T13:31:15Z)
• HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentation [47.271784693700845]
  We propose a novel hybrid pseudo-labeling framework for unsupervised event-based semantic segmentation, HPL-ESS, to alleviate the influence of noisy pseudo labels. Our proposed method outperforms existing state-of-the-art methods by a large margin on the DSEC-Semantic dataset.
  arXiv  Detail & Related papers  (2024-03-25T14:02:33Z)
• Semi-Supervised Semantic Segmentation via Marginal Contextual Information [13.721552758997225]
  We present a novel confidence refinement scheme that enhances pseudo labels in semi-supervised semantic segmentation. Our method, named S4MC, increases the amount of unlabeled data used during training while maintaining the quality of the pseudo labels.
  arXiv  Detail & Related papers  (2023-08-26T15:02:00Z)
• Weakly Supervised 3D Instance Segmentation without Instance-level Annotations [57.615325809883636]
  3D semantic scene understanding tasks have achieved great success with the emergence of deep learning, but often require a huge amount of manually annotated training data. We propose the first weakly-supervised 3D instance segmentation method that only requires categorical semantic labels as supervision. By generating pseudo instance labels from categorical semantic labels, our designed approach can also assist existing methods for learning 3D instance segmentation at reduced annotation cost.
  arXiv  Detail & Related papers  (2023-08-03T12:30:52Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning [59.64695628433855]
  We propose a novel cross-modality weakly supervised method for 3D segmentation, incorporating complementary information from unlabeled images. Basically, we design a dual-branch network equipped with an active labeling strategy, to maximize the power of tiny parts of labels. Our method even outperforms the state-of-the-art fully supervised competitors with less than 1% actively selected annotations.
  arXiv  Detail & Related papers  (2022-09-16T07:59:04Z)
• Pseudo Pixel-level Labeling for Images with Evolving Content [5.573543601558405]
  We propose a pseudo-pixel-level label generation technique to reduce the amount of effort for manual annotation of images. We train two semantic segmentation models with VGG and ResNet backbones on images labeled using our pseudo labeling method and those of a state-of-the-art method. The results indicate that using our pseudo-labels instead of those generated using the state-of-the-art method in the training process improves the mean-IoU and the frequency-weighted-IoU of the VGG and ResNet-based semantic segmentation models by 3.36%, 2.58%, 10
  arXiv  Detail & Related papers  (2021-05-20T18:14:19Z)
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation [78.36781565047656]
  We propose "One Thing One Click," meaning that the annotator only needs to label one point per object. We iteratively conduct the training and label propagation, facilitated by a graph propagation module. Our results are also comparable to those of the fully supervised counterparts.
  arXiv  Detail & Related papers  (2021-04-06T02:27:25Z)
• Semantics through Time: Semi-supervised Segmentation of Aerial Videos with Iterative Label Propagation [16.478668565965243]
  This paper makes an important step towards automatic annotation by introducing SegProp. SegProp is a novel iterative flow-based method, with a direct connection to spectral clustering in space and time. We introduce Ruralscapes, a new dataset with high resolution (4K) images and manually-annotated dense labels every 50 frames. Our novel SegProp automatically annotates the remaining unlabeled 98% of frames with an accuracy exceeding 90%.
  arXiv  Detail & Related papers  (2020-10-02T15:15:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.