LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories

• URL: http://arxiv.org/abs/2311.12174v1
• Date: Mon, 20 Nov 2023 20:40:24 GMT
• Title: LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories
• Authors: Silvan Weder, Hermann Blum, Francis Engelmann, Marc Pollefeys
• Abstract summary: This work introduces a fully automated 2D/3D labeling framework that can generate labels for RGB-D scans at equal (or better) level of accuracy. We demonstrate the effectiveness of our LabelMaker pipeline by generating significantly better labels for the ScanNet datasets and automatically labelling the previously unlabeled ARKitScenes dataset.
• Score: 59.14011485494713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Semantic annotations are indispensable to train or evaluate perception models, yet very costly to acquire. This work introduces a fully automated 2D/3D labeling framework that, without any human intervention, can generate labels for RGB-D scans at equal (or better) level of accuracy than comparable manually annotated datasets such as ScanNet. Our approach is based on an ensemble of state-of-the-art segmentation models and 3D lifting through neural rendering. We demonstrate the effectiveness of our LabelMaker pipeline by generating significantly better labels for the ScanNet datasets and automatically labelling the previously unlabeled ARKitScenes dataset. Code and models are available at https://labelmaker.org

Related papers

• ARKit LabelMaker: A New Scale for Indoor 3D Scene Understanding [51.509115746992165]
  We introduce ARKit LabelMaker, the first large-scale, real-world 3D dataset with dense semantic annotations. We also push forward the state-of-the-art performance on ScanNet and ScanNet200 dataset with prevalent 3D semantic segmentation models.
  arXiv  Detail & Related papers  (2024-10-17T14:44:35Z)
• Segment, Lift and Fit: Automatic 3D Shape Labeling from 2D Prompts [50.181870446016376]
  This paper proposes an algorithm for automatically labeling 3D objects from 2D point or box prompts. Unlike previous arts, our auto-labeler predicts 3D shapes instead of bounding boxes and does not require training on a specific dataset.
  arXiv  Detail & Related papers  (2024-07-16T04:53:28Z)
• You Only Need One Thing One Click: Self-Training for Weakly Supervised 3D Scene Understanding [107.06117227661204]
  We propose One Thing One Click'', meaning that the annotator only needs to label one point per object. We iteratively conduct the training and label propagation, facilitated by a graph propagation module. Our model can be compatible to 3D instance segmentation equipped with a point-clustering strategy.
  arXiv  Detail & Related papers  (2023-03-26T13:57:00Z)
• 3D-aware Conditional Image Synthesis [76.68701564600998]
  pix2pix3D is a 3D-aware conditional generative model for controllable photorealistic image synthesis. We build an interactive system that allows users to edit the label map from any viewpoint and generate outputs accordingly.
  arXiv  Detail & Related papers  (2023-02-16T18:59:53Z)
• From CAD models to soft point cloud labels: An automatic annotation pipeline for cheaply supervised 3D semantic segmentation [0.0]
  We propose a fully automatic annotation scheme that takes a raw 3D point cloud with a set of fitted CAD models as input and outputs convincing point-wise labels. Compared with manual annotations, we show that our automatic labels are accurate while drastically reducing the annotation time. We evaluate the label quality and segmentation performance of PointNet++ on a dataset of real industrial point clouds and Scan2CAD, a public dataset of indoor scenes.
  arXiv  Detail & Related papers  (2023-02-06T20:33:16Z)
• iLabel: Interactive Neural Scene Labelling [20.63756683450811]
  Joint representation of geometry, colour and semantics using a 3D neural field enables accurate dense labelling from ultra-sparse interactions. Our iLabel system requires no training data, yet can densely label scenes more accurately than standard methods. It works in an 'open set' manner, with semantic classes defined on the fly by the user.
  arXiv  Detail & Related papers  (2021-11-29T15:49:20Z)
• DeepScanner: a Robotic System for Automated 2D Object Dataset Collection with Annotations [4.0423807111935295]
  We describe the possibility of automated dataset collection using an articulated robot. The proposed technology reduces the number of pixel errors on a polygonal dataset and the time spent on manual labeling of 2D objects.
  arXiv  Detail & Related papers  (2021-08-05T12:21:18Z)
• Manual-Label Free 3D Detection via An Open-Source Simulator [50.74299948748722]
  We propose a manual-label free 3D detection algorithm that leverages the CARLA simulator to generate a large amount of self-labeled training samples. Domain Adaptive VoxelNet (DA-VoxelNet) can cross the distribution gap from the synthetic data to the real scenario. Experimental results show that the proposed unsupervised DA 3D detector can achieve 76.66% and 56.64% mAP on KITTI evaluation set.
  arXiv  Detail & Related papers  (2020-11-16T08:29:01Z)
• Rapid Pose Label Generation through Sparse Representation of Unknown Objects [7.32172860877574]
  This work presents an approach for rapidly generating real-world, pose-annotated RGB-D data for unknown objects. We first source minimalistic labelings of an ordered set of arbitrarily chosen keypoints over a set of RGB-D videos. By solving an optimization problem, we combine these labels under a world frame to recover a sparse, keypoint-based representation of the object.
  arXiv  Detail & Related papers  (2020-11-07T15:14:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.