An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains

• URL: http://arxiv.org/abs/2402.17562v1
• Date: Tue, 27 Feb 2024 15:02:17 GMT
• Title: An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains
• Authors: George Eskandar, Chongzhe Zhang, Abhishek Kaushik, Karim Guirguis, Mohamed Sayed, Bin Yang
• Abstract summary: 3D Object Detectors (3D-OD) are crucial for understanding the environment in many robotic tasks, especially autonomous driving. Here we focus on the details of 3D-ODs, focusing on fundamental factors that influence robustness prior to domain adaptation. Our main findings are: (1) transformer backbones with local point features are more robust than 3D CNNs, (2) test-time anchor size adjustment is crucial for adaptation across geographical locations, significantly boosting scores without retraining, and (4) surprisingly, robustness to bad weather is improved when training directly on more clean weather data than on training with bad weather data.
• Score: 6.4288046828223315
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D Object Detectors (3D-OD) are crucial for understanding the environment in many robotic tasks, especially autonomous driving. Including 3D information via Lidar sensors improves accuracy greatly. However, such detectors perform poorly on domains they were not trained on, i.e. different locations, sensors, weather, etc., limiting their reliability in safety-critical applications. There exist methods to adapt 3D-ODs to these domains; however, these methods treat 3D-ODs as a black box, neglecting underlying architectural decisions and source-domain training strategies. Instead, we dive deep into the details of 3D-ODs, focusing our efforts on fundamental factors that influence robustness prior to domain adaptation. We systematically investigate four design choices (and the interplay between them) often overlooked in 3D-OD robustness and domain adaptation: architecture, voxel encoding, data augmentations, and anchor strategies. We assess their impact on the robustness of nine state-of-the-art 3D-ODs across six benchmarks encompassing three types of domain gaps - sensor type, weather, and location. Our main findings are: (1) transformer backbones with local point features are more robust than 3D CNNs, (2) test-time anchor size adjustment is crucial for adaptation across geographical locations, significantly boosting scores without retraining, (3) source-domain augmentations allow the model to generalize to low-resolution sensors, and (4) surprisingly, robustness to bad weather is improved when training directly on more clean weather data than on training with bad weather data. We outline our main conclusions and findings to provide practical guidance on developing more robust 3D-ODs.

Related papers

• Revisiting Cross-Domain Problem for LiDAR-based 3D Object Detection [5.149095033945412]
  We deeply analyze the cross-domain performance of the state-of-the-art models. We observe that most models will overfit the training domains and it is challenging to adapt them to other domains directly. We propose additional evaluation metrics -- the side-view and front-view AP -- to better analyze the core issues of the methods' heavy drops in accuracy levels.
  arXiv  Detail & Related papers  (2024-08-22T19:52:44Z)
• Exploring Domain Shift on Radar-Based 3D Object Detection Amidst Diverse Environmental Conditions [15.767261586617746]
  This study delves into the often-overlooked yet crucial issue of domain shift in 4D radar-based object detection. Our findings highlight distinct domain shifts across various weather scenarios, revealing unique dataset sensitivities. transitioning between different road types, especially from highways to urban settings, introduces notable domain shifts.
  arXiv  Detail & Related papers  (2024-08-13T09:55:38Z)
• DiffuBox: Refining 3D Object Detection with Point Diffusion [74.01759893280774]
  We introduce a novel diffusion-based box refinement approach to ensure robust 3D object detection and localization. We evaluate this approach under various domain adaptation settings, and our results reveal significant improvements across different datasets.
  arXiv  Detail & Related papers  (2024-05-25T03:14:55Z)
• UADA3D: Unsupervised Adversarial Domain Adaptation for 3D Object Detection with Sparse LiDAR and Large Domain Gaps [2.79552147676281]
  We introduce Unsupervised Adversarial Domain Adaptation for 3D Object Detection (UADA3D) We demonstrate its efficacy in various adaptation scenarios, showing significant improvements in both self-driving car and mobile robot domains. Our code is open-source and will be available soon.
  arXiv  Detail & Related papers  (2024-03-26T12:08:14Z)
• AdvMono3D: Advanced Monocular 3D Object Detection with Depth-Aware Robust Adversarial Training [64.14759275211115]
  We propose a depth-aware robust adversarial training method for monocular 3D object detection, dubbed DART3D. Our adversarial training approach capitalizes on the inherent uncertainty, enabling the model to significantly improve its robustness against adversarial attacks.
  arXiv  Detail & Related papers  (2023-09-03T07:05:32Z)
• MS3D++: Ensemble of Experts for Multi-Source Unsupervised Domain Adaptation in 3D Object Detection [12.005805403222354]
  Deploying 3D detectors in unfamiliar domains has been demonstrated to result in a significant 70-90% drop in detection rate. We introduce MS3D++, a self-training framework for multi-source unsupervised domain adaptation in 3D object detection. MS3D++ generates high-quality pseudo-labels, allowing 3D detectors to achieve high performance on a range of lidar types.
  arXiv  Detail & Related papers  (2023-08-11T07:56:10Z)
• MS3D: Leveraging Multiple Detectors for Unsupervised Domain Adaptation in 3D Object Detection [7.489722641968593]
  Multi-Source 3D (MS3D) is a new self-training pipeline for unsupervised domain adaptation in 3D object detection. Our proposed Kernel-Density Estimation (KDE) Box Fusion method fuses box proposals from multiple domains to obtain pseudo-labels. MS3D exhibits greater robustness to domain shift and produces accurate pseudo-labels over greater distances.
  arXiv  Detail & Related papers  (2023-04-05T13:29:21Z)
• Robo3D: Towards Robust and Reliable 3D Perception against Corruptions [58.306694836881235]
  We present Robo3D, the first comprehensive benchmark heading toward probing the robustness of 3D detectors and segmentors under out-of-distribution scenarios. We consider eight corruption types stemming from severe weather conditions, external disturbances, and internal sensor failure. We propose a density-insensitive training framework along with a simple flexible voxelization strategy to enhance the model resiliency.
  arXiv  Detail & Related papers  (2023-03-30T17:59:17Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency [90.71745178767203]
  Deep learning-based 3D object detection has achieved unprecedented success with the advent of large-scale autonomous driving datasets. Existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world. We study a more realistic setting, unsupervised 3D domain adaptive detection, which only utilizes source domain annotations.
  arXiv  Detail & Related papers  (2021-07-23T17:19:23Z)
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D ObjectDetection [78.71826145162092]
  We present a new domain adaptive self-training pipeline, named ST3D, for unsupervised domain adaptation on 3D object detection from point clouds. Our ST3D achieves state-of-the-art performance on all evaluated datasets and even surpasses fully supervised results on KITTI 3D object detection benchmark.
  arXiv  Detail & Related papers  (2021-03-09T10:51:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.