A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation

• URL: http://arxiv.org/abs/2405.19035v1
• Date: Wed, 29 May 2024 12:23:29 GMT
• Title: A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation
• Authors: Niclas Vödisch, Kürsat Petek, Markus Käppeler, Abhinav Valada, Wolfram Burgard,
• Abstract summary: Key challenge for the widespread application of learning-based models for robotic perception is to significantly reduce the required amount of annotated training data. We exploit the groundwork paved by visual foundation models to train two lightweight network heads for semantic segmentation and object boundary detection. We demonstrate that PASTEL significantly outperforms previous methods for label-efficient segmentation even when using fewer annotations.
• Score: 22.440065488051047
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A key challenge for the widespread application of learning-based models for robotic perception is to significantly reduce the required amount of annotated training data while achieving accurate predictions. This is essential not only to decrease operating costs but also to speed up deployment time. In this work, we address this challenge for PAnoptic SegmenTation with fEw Labels (PASTEL) by exploiting the groundwork paved by visual foundation models. We leverage descriptive image features from such a model to train two lightweight network heads for semantic segmentation and object boundary detection, using very few annotated training samples. We then merge their predictions via a novel fusion module that yields panoptic maps based on normalized cut. To further enhance the performance, we utilize self-training on unlabeled images selected by a feature-driven similarity scheme. We underline the relevance of our approach by employing PASTEL to important robot perception use cases from autonomous driving and agricultural robotics. In extensive experiments, we demonstrate that PASTEL significantly outperforms previous methods for label-efficient segmentation even when using fewer annotations. The code of our work is publicly available at http://pastel.cs.uni-freiburg.de.

Related papers

• Task Specific Pretraining with Noisy Labels for Remote Sensing Image Segmentation [18.598405597933752]
  Self-supervision provides remote sensing a tool to reduce the amount of exact, human-crafted geospatial annotations. In this work, we propose to exploit noisy semantic segmentation maps for model pretraining. The results from two datasets indicate the effectiveness of task-specific supervised pretraining with noisy labels.
  arXiv  Detail & Related papers  (2024-02-25T18:01:42Z)
• Few-Shot Panoptic Segmentation With Foundation Models [23.231014713335664]
  We propose to leverage task-agnostic image features to enable few-shot panoptic segmentation by presenting Segmenting Panoptic Information with Nearly 0 labels (SPINO) In detail, our method combines a DINOv2 backbone with lightweight network heads for semantic segmentation and boundary estimation. We show that our approach, albeit being trained with only ten annotated images, predicts high-quality pseudo-labels that can be used with any existing panoptic segmentation method.
  arXiv  Detail & Related papers  (2023-09-19T16:09:01Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Active Self-Training for Weakly Supervised 3D Scene Semantic Segmentation [17.27850877649498]
  We introduce a method for weakly supervised segmentation of 3D scenes that combines self-training and active learning. We demonstrate that our approach leads to an effective method that provides improvements in scene segmentation over previous works and baselines.
  arXiv  Detail & Related papers  (2022-09-15T06:00:25Z)
• Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition [98.25592165484737]
  We propose a more effective pseudo-labeling scheme, called Cross-Model Pseudo-Labeling (CMPL) CMPL achieves $17.6%$ and $25.1%$ Top-1 accuracy on Kinetics-400 and UCF-101 using only the RGB modality and $1%$ labeled data, respectively.
  arXiv  Detail & Related papers  (2021-12-17T18:59:41Z)
• Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets [90.61266099147053]
  We investigate efficient annotation strategies for collecting multi-class classification labels for a large collection of images. We propose modifications and best practices aimed at minimizing human labeling effort. Simulated experiments on a 125k image subset of the ImageNet100 show that it can be annotated to 80% top-1 accuracy with 0.35 annotations per image on average.
  arXiv  Detail & Related papers  (2021-04-26T16:29:32Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)
• Improving Semantic Segmentation via Self-Training [75.07114899941095]
  We show that we can obtain state-of-the-art results using a semi-supervised approach, specifically a self-training paradigm. We first train a teacher model on labeled data, and then generate pseudo labels on a large set of unlabeled data. Our robust training framework can digest human-annotated and pseudo labels jointly and achieve top performances on Cityscapes, CamVid and KITTI datasets.
  arXiv  Detail & Related papers  (2020-04-30T17:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.