Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement

• URL: http://arxiv.org/abs/2203.05238v1
• Date: Thu, 10 Mar 2022 08:51:32 GMT
• Title: Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement
• Authors: Xiuwei Xu, Yifan Wang, Yu Zheng, Yongming Rao, Jiwen Lu, Jie Zhou
• Abstract summary: We propose a weakly-supervised approach for 3D object detection, which makes it possible to train strong 3D detector with position-level annotations. Our method, namely Back to Reality (BR), makes use of synthetic 3D shapes to convert the weak labels into fully-annotated virtual scenes. With less than 5% of the labeling labor, we achieve comparable detection performance with some popular fully-supervised approaches on the widely used ScanNet dataset.
• Score: 93.77156425817178
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a weakly-supervised approach for 3D object detection, which makes it possible to train strong 3D detector with position-level annotations (i.e. annotations of object centers). In order to remedy the information loss from box annotations to centers, our method, namely Back to Reality (BR), makes use of synthetic 3D shapes to convert the weak labels into fully-annotated virtual scenes as stronger supervision, and in turn utilizes the perfect virtual labels to complement and refine the real labels. Specifically, we first assemble 3D shapes into physically reasonable virtual scenes according to the coarse scene layout extracted from position-level annotations. Then we go back to reality by applying a virtual-to-real domain adaptation method, which refine the weak labels and additionally supervise the training of detector with the virtual scenes. Furthermore, we propose a more challenging benckmark for indoor 3D object detection with more diversity in object sizes to better show the potential of BR. With less than 5% of the labeling labor, we achieve comparable detection performance with some popular fully-supervised approaches on the widely used ScanNet dataset. Code is available at: https://github.com/xuxw98/BackToReality

Related papers

• VSRD: Instance-Aware Volumetric Silhouette Rendering for Weakly Supervised 3D Object Detection [11.061100776969383]
  Monocular 3D object detection poses a significant challenge in 3D scene understanding. Existing methods heavily rely on supervised learning using abundant 3D labels. We propose a novel weakly supervised 3D object detection framework named VSRD.
  arXiv  Detail & Related papers  (2024-03-29T20:43:55Z)
• Point-DETR3D: Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection [32.86369670395974]
  We introduce Point-DETR3D, a teacher-student framework for weakly semi-supervised 3D detection. With only 5% of labeled data, Point-DETR3D achieves over 90% performance of its fully supervised counterpart.
  arXiv  Detail & Related papers  (2024-03-22T16:11:29Z)
• Weakly Supervised 3D Object Detection via Multi-Level Visual Guidance [72.6809373191638]
  We propose a framework to study how to leverage constraints between 2D and 3D domains without requiring any 3D labels. Specifically, we design a feature-level constraint to align LiDAR and image features based on object-aware regions. Second, the output-level constraint is developed to enforce the overlap between 2D and projected 3D box estimations. Third, the training-level constraint is utilized by producing accurate and consistent 3D pseudo-labels that align with the visual data.
  arXiv  Detail & Related papers  (2023-12-12T18:57:25Z)
• Weakly Supervised 3D Object Detection with Multi-Stage Generalization [62.96670547848691]
  We introduce BA$2$-Det, encompassing pseudo label generation and multi-stage generalization. We develop three stages of generalization: progressing from complete to partial, static to dynamic, and close to distant. BA$2$-Det can achieve a 20% relative improvement on the KITTI dataset.
  arXiv  Detail & Related papers  (2023-06-08T17:58:57Z)
• CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection [57.44434974289945]
  We propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework. Our framework takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene. In addition to 3D object detection, we investigate the effectiveness of our framework for the problem of 3D object counting.
  arXiv  Detail & Related papers  (2022-09-13T05:26:09Z)
• ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection [78.71826145162092]
  We present a self-training method, named ST3D++, with a holistic pseudo label denoising pipeline for unsupervised domain adaptation on 3D object detection. We equip the pseudo label generation process with a hybrid quality-aware triplet memory to improve the quality and stability of generated pseudo labels. In the model training stage, we propose a source data assisted training strategy and a curriculum data augmentation policy.
  arXiv  Detail & Related papers  (2021-08-15T07:49:06Z)
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D ObjectDetection [78.71826145162092]
  We present a new domain adaptive self-training pipeline, named ST3D, for unsupervised domain adaptation on 3D object detection from point clouds. Our ST3D achieves state-of-the-art performance on all evaluated datasets and even surpasses fully supervised results on KITTI 3D object detection benchmark.
  arXiv  Detail & Related papers  (2021-03-09T10:51:24Z)
• Learning Monocular 3D Vehicle Detection without 3D Bounding Box Labels [0.09558392439655011]
  Training of 3D object detectors requires datasets with 3D bounding box labels for supervision that have to be generated by hand-labeling. We propose a network architecture and training procedure for learning monocular 3D object detection without 3D bounding box labels. We evaluate the proposed algorithm on the real-world KITTI dataset and achieve promising performance in comparison to state-of-the-art methods requiring 3D bounding box labels for training.
  arXiv  Detail & Related papers  (2020-10-07T16:24:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.