Open-CRB: Towards Open World Active Learning for 3D Object Detection

• URL: http://arxiv.org/abs/2310.10391v2
• Date: Mon, 23 Sep 2024 04:48:30 GMT
• Title: Open-CRB: Towards Open World Active Learning for 3D Object Detection
• Authors: Zhuoxiao Chen, Yadan Luo, Zixin Wang, Zijian Wang, Xin Yu, Zi Huang,
• Abstract summary: LiDAR-based 3D object detection has recently seen significant advancements through active learning (AL) In real-world deployments where streaming point clouds may include unknown or novel objects, the ability of current AL methods to capture such objects remains unexplored. This paper investigates a more practical and challenging research task: Open World Active Learning for 3D Object Detection (OWAL-3D)
• Score: 40.80953254074535
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: LiDAR-based 3D object detection has recently seen significant advancements through active learning (AL), attaining satisfactory performance by training on a small fraction of strategically selected point clouds. However, in real-world deployments where streaming point clouds may include unknown or novel objects, the ability of current AL methods to capture such objects remains unexplored. This paper investigates a more practical and challenging research task: Open World Active Learning for 3D Object Detection (OWAL-3D), aimed at acquiring informative point clouds with new concepts. To tackle this challenge, we propose a simple yet effective strategy called Open Label Conciseness (OLC), which mines novel 3D objects with minimal annotation costs. Our empirical results show that OLC successfully adapts the 3D detection model to the open world scenario with just a single round of selection. Any generic AL policy can then be integrated with the proposed OLC to efficiently address the OWAL-3D problem. Based on this, we introduce the Open-CRB framework, which seamlessly integrates OLC with our preliminary AL method, CRB, designed specifically for 3D object detection. We develop a comprehensive codebase for easy reproducing and future research, supporting 15 baseline methods (\textit{i.e.}, active learning, out-of-distribution detection and open world detection), 2 types of modern 3D detectors (\textit{i.e.}, one-stage SECOND and two-stage PV-RCNN) and 3 benchmark 3D datasets (\textit{i.e.}, KITTI, nuScenes and Waymo). Extensive experiments evidence that the proposed Open-CRB demonstrates superiority and flexibility in recognizing both novel and known classes with very limited labeling costs, compared to state-of-the-art baselines. Source code is available at \url{https://github.com/Luoyadan/CRB-active-3Ddet/tree/Open-CRB}.

Related papers

• Training an Open-Vocabulary Monocular 3D Object Detection Model without 3D Data [57.53523870705433]
  We propose a novel open-vocabulary monocular 3D object detection framework, dubbed OVM3D-Det. OVM3D-Det does not require high-precision LiDAR or 3D sensor data for either input or generating 3D bounding boxes. It employs open-vocabulary 2D models and pseudo-LiDAR to automatically label 3D objects in RGB images, fostering the learning of open-vocabulary monocular 3D detectors.
  arXiv  Detail & Related papers  (2024-11-23T21:37:21Z)
• Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environments [67.83787474506073]
  We tackle the limitations of current LiDAR-based 3D object detection systems. We introduce a universal textscFind n' Propagate approach for 3D OV tasks. We achieve up to a 3.97-fold increase in Average Precision (AP) for novel object classes.
  arXiv  Detail & Related papers  (2024-03-20T12:51:30Z)
• Open-Pose 3D Zero-Shot Learning: Benchmark and Challenges [23.663199578392447]
  We propose a more realistic and challenging scenario named open-pose 3D zero-shot classification. First, we revisit the current research on 3D zero-shot classification. We propose two benchmark datasets specifically designed for the open-pose setting.
  arXiv  Detail & Related papers  (2023-12-12T07:52:33Z)
• Exploring Active 3D Object Detection from a Generalization Perspective [58.597942380989245]
  Uncertainty-based active learning policies fail to balance the trade-off between point cloud informativeness and box-level annotation costs. We propose textscCrb, which hierarchically filters out the point clouds of redundant 3D bounding box labels. Experiments show that the proposed approach outperforms existing active learning strategies.
  arXiv  Detail & Related papers  (2023-01-23T02:43:03Z)
• Towards Open Set 3D Learning: A Benchmark on Object Point Clouds [17.145309633743747]
  This paper provides the first broad study on Open Set 3D learning. We introduce a novel testbed with settings of increasing difficulty in terms of category semantic shift. We investigate the related out-of-distribution and Open Set 2D literature to understand if and how their most recent approaches are effective on 3D data.
  arXiv  Detail & Related papers  (2022-07-23T17:00:45Z)
• Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving [45.405303803618]
  We investigate diversity-based active learning (AL) as a potential solution to alleviate the annotation burden. We propose a novel acquisition function that enforces spatial and temporal diversity in the selected samples. We demonstrate the effectiveness of the proposed method on the nuScenes dataset and show that it outperforms existing AL strategies significantly.
  arXiv  Detail & Related papers  (2022-05-16T14:21:30Z)
• From Multi-View to Hollow-3D: Hallucinated Hollow-3D R-CNN for 3D Object Detection [101.20784125067559]
  We propose a new architecture, namely Hallucinated Hollow-3D R-CNN, to address the problem of 3D object detection. In our approach, we first extract the multi-view features by sequentially projecting the point clouds into the perspective view and the bird-eye view. The 3D objects are detected via a box refinement module with a novel Hierarchical Voxel RoI Pooling operation.
  arXiv  Detail & Related papers  (2021-07-30T02:00:06Z)
• Weakly Supervised 3D Object Detection from Point Clouds [27.70180601788613]
  3D object detection aims to detect and localize the 3D bounding boxes of objects belonging to specific classes. Existing 3D object detectors rely on annotated 3D bounding boxes during training, while these annotations could be expensive to obtain and only accessible in limited scenarios. We propose VS3D, a framework for weakly supervised 3D object detection from point clouds without using any ground truth 3D bounding box for training.
  arXiv  Detail & Related papers  (2020-07-28T03:30:11Z)
• SESS: Self-Ensembling Semi-Supervised 3D Object Detection [138.80825169240302]
  We propose SESS, a self-ensembling semi-supervised 3D object detection framework. Specifically, we design a thorough perturbation scheme to enhance generalization of the network on unlabeled and new unseen data. Our SESS achieves competitive performance compared to the state-of-the-art fully-supervised method by using only 50% labeled data.
  arXiv  Detail & Related papers  (2019-12-26T08:48:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.