Robo3D: Towards Robust and Reliable 3D Perception against Corruptions

• URL: http://arxiv.org/abs/2303.17597v4
• Date: Sat, 2 Sep 2023 23:52:19 GMT
• Title: Robo3D: Towards Robust and Reliable 3D Perception against Corruptions
• Authors: Lingdong Kong and Youquan Liu and Xin Li and Runnan Chen and Wenwei Zhang and Jiawei Ren and Liang Pan and Kai Chen and Ziwei Liu
• Abstract summary: We present Robo3D, the first comprehensive benchmark heading toward probing the robustness of 3D detectors and segmentors under out-of-distribution scenarios. We consider eight corruption types stemming from severe weather conditions, external disturbances, and internal sensor failure. We propose a density-insensitive training framework along with a simple flexible voxelization strategy to enhance the model resiliency.
• Score: 58.306694836881235
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The robustness of 3D perception systems under natural corruptions from environments and sensors is pivotal for safety-critical applications. Existing large-scale 3D perception datasets often contain data that are meticulously cleaned. Such configurations, however, cannot reflect the reliability of perception models during the deployment stage. In this work, we present Robo3D, the first comprehensive benchmark heading toward probing the robustness of 3D detectors and segmentors under out-of-distribution scenarios against natural corruptions that occur in real-world environments. Specifically, we consider eight corruption types stemming from severe weather conditions, external disturbances, and internal sensor failure. We uncover that, although promising results have been progressively achieved on standard benchmarks, state-of-the-art 3D perception models are at risk of being vulnerable to corruptions. We draw key observations on the use of data representations, augmentation schemes, and training strategies, that could severely affect the model's performance. To pursue better robustness, we propose a density-insensitive training framework along with a simple flexible voxelization strategy to enhance the model resiliency. We hope our benchmark and approach could inspire future research in designing more robust and reliable 3D perception models. Our robustness benchmark suite is publicly available.

Related papers

• Enhanced Model Robustness to Input Corruptions by Per-corruption Adaptation of Normalization Statistics [22.876222327262596]
  We introduce Per-corruption Adaptation of Normalization statistics (PAN) to enhance the model robustness of vision systems. Our approach entails three key components: (i) a corruption type identification module, (ii) dynamic adjustment of normalization layer statistics based on identified corruption type, and (iii) real-time update of these statistics according to input data.
  arXiv  Detail & Related papers  (2024-07-08T23:20:18Z)
• PoseBench: Benchmarking the Robustness of Pose Estimation Models under Corruptions [57.871692507044344]
  Pose estimation aims to accurately identify anatomical keypoints in humans and animals using monocular images. Current models are typically trained and tested on clean data, potentially overlooking the corruption during real-world deployment. We introduce PoseBench, a benchmark designed to evaluate the robustness of pose estimation models against real-world corruption.
  arXiv  Detail & Related papers  (2024-06-20T14:40:17Z)
• Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving [55.93813178692077]
  We present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms. We assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction. Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data.
  arXiv  Detail & Related papers  (2024-05-27T17:59:39Z)
• Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding [55.32861154245772]
  Calib3D is a pioneering effort to benchmark and scrutinize the reliability of 3D scene understanding models. We evaluate 28 state-of-the-art models across 10 diverse 3D datasets. We introduce DeptS, a novel depth-aware scaling approach aimed at enhancing 3D model calibration.
  arXiv  Detail & Related papers  (2024-03-25T17:59:59Z)
• RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions [7.359657743276515]
  We introduce a comprehensive robustness test suite, RoboDepth, spanning 18 corruptions spanning three categories. We benchmark 42 depth estimation models across indoor and outdoor scenes to assess their resilience to these corruptions. Our findings underscore that, in the absence of a dedicated robustness evaluation framework, many leading depth estimation models may be susceptible to typical corruptions.
  arXiv  Detail & Related papers  (2023-10-23T17:59:59Z)
• A Survey on the Robustness of Computer Vision Models against Common Corruptions [3.6486148851646063]
  Computer vision models are susceptible to changes in input images caused by sensor errors or extreme imaging environments. These corruptions can significantly hinder the reliability of these models when deployed in real-world scenarios. We present a comprehensive overview of methods that improve the robustness of computer vision models against common corruptions.
  arXiv  Detail & Related papers  (2023-05-10T10:19:31Z)
• Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving [44.753797839280516]
  Existing 3D detectors lack robustness to real-world corruptions caused by adverse weathers, sensor noises, etc. We benchmark 27 types of common corruptions for both LiDAR and camera inputs considering real-world driving scenarios. We conduct large-scale experiments on 24 diverse 3D object detection models to evaluate their robustness.
  arXiv  Detail & Related papers  (2023-03-20T11:45:54Z)
• On the Robustness of Quality Measures for GANs [136.18799984346248]
  This work evaluates the robustness of quality measures of generative models such as Inception Score (IS) and Fr'echet Inception Distance (FID) We show that such metrics can also be manipulated by additive pixel perturbations.
  arXiv  Detail & Related papers  (2022-01-31T06:43:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.