Sample-adaptive Augmentation for Point Cloud Recognition Against Real-world Corruptions

• URL: http://arxiv.org/abs/2309.10431v1
• Date: Tue, 19 Sep 2023 08:46:43 GMT
• Title: Sample-adaptive Augmentation for Point Cloud Recognition Against Real-world Corruptions
• Authors: Jie Wang, Lihe Ding, Tingfa Xu, Shaocong Dong, Xinli Xu, Long Bai, Jianan Li
• Abstract summary: We propose an alternative to make sample-adaptive transformations based on the structure of the sample, named as AdaptPoint. A discriminator is utilized to prevent the generation of excessive corruption that deviates from the original data distribution. Experiments show that our method achieves state-of-the-art results on multiple corruption benchmarks, including ModelNet-C, our ScanObjectNN-C, and ShapeNet-C.
• Score: 20.95456179904285
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robust 3D perception under corruption has become an essential task for the realm of 3D vision. While current data augmentation techniques usually perform random transformations on all point cloud objects in an offline way and ignore the structure of the samples, resulting in over-or-under enhancement. In this work, we propose an alternative to make sample-adaptive transformations based on the structure of the sample to cope with potential corruption via an auto-augmentation framework, named as AdaptPoint. Specially, we leverage a imitator, consisting of a Deformation Controller and a Mask Controller, respectively in charge of predicting deformation parameters and producing a per-point mask, based on the intrinsic structural information of the input point cloud, and then conduct corruption simulations on top. Then a discriminator is utilized to prevent the generation of excessive corruption that deviates from the original data distribution. In addition, a perception-guidance feedback mechanism is incorporated to guide the generation of samples with appropriate difficulty level. Furthermore, to address the paucity of real-world corrupted point cloud, we also introduce a new dataset ScanObjectNN-C, that exhibits greater similarity to actual data in real-world environments, especially when contrasted with preceding CAD datasets. Experiments show that our method achieves state-of-the-art results on multiple corruption benchmarks, including ModelNet-C, our ScanObjectNN-C, and ShapeNet-C.

Related papers

• 3D Adversarial Augmentations for Robust Out-of-Domain Predictions [115.74319739738571]
  We focus on improving the generalization to out-of-domain data. We learn a set of vectors that deform the objects in an adversarial fashion. We perform adversarial augmentation by applying the learned sample-independent vectors to the available objects when training a model.
  arXiv  Detail & Related papers  (2023-08-29T17:58:55Z)
• Robo3D: Towards Robust and Reliable 3D Perception against Corruptions [58.306694836881235]
  We present Robo3D, the first comprehensive benchmark heading toward probing the robustness of 3D detectors and segmentors under out-of-distribution scenarios. We consider eight corruption types stemming from severe weather conditions, external disturbances, and internal sensor failure. We propose a density-insensitive training framework along with a simple flexible voxelization strategy to enhance the model resiliency.
  arXiv  Detail & Related papers  (2023-03-30T17:59:17Z)
• PCV: A Point Cloud-Based Network Verifier [8.239631885389382]
  We describe a point cloud-based network verifier that successfully deals state of the art 3D PointNet. We calculate the impact on model accuracy versus property factor and can test PointNet network's robustness against a small collection of perturbing input states.
  arXiv  Detail & Related papers  (2023-01-27T15:58:54Z)
• Point-DAE: Denoising Autoencoders for Self-supervised Point Cloud Learning [58.48149060125912]
  We explore a more general denoising autoencoder for point cloud learning (Point-DAE) by investigating more types of corruptions beyond masking. Specifically, we degrade the point cloud with certain corruptions as input, and learn an encoder-decoder model to reconstruct the original point cloud from its corrupted version.
  arXiv  Detail & Related papers  (2022-11-13T08:02:03Z)
• Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions [38.89370166717221]
  We present ModelNet40-C, the first comprehensive benchmark on 3D point cloud corruption robustness. Our evaluation shows a significant gap between the performances on ModelNet40 and ModelNet40-C for state-of-the-art (SOTA) models. We unveil that Transformer-based architectures with proper training recipes achieve the strongest robustness.
  arXiv  Detail & Related papers  (2022-01-28T18:01:42Z)
• 3D-VField: Learning to Adversarially Deform Point Clouds for Robust 3D Object Detection [111.32054128362427]
  In safety-critical settings, robustness on out-of-distribution and long-tail samples is fundamental to circumvent dangerous issues. We substantially improve the generalization of 3D object detectors to out-of-domain data by taking into account deformed point clouds during training. We propose and share open source CrashD: a synthetic dataset of realistic damaged and rare cars.
  arXiv  Detail & Related papers  (2021-12-09T08:50:54Z)
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D ObjectDetection [78.71826145162092]
  We present a new domain adaptive self-training pipeline, named ST3D, for unsupervised domain adaptation on 3D object detection from point clouds. Our ST3D achieves state-of-the-art performance on all evaluated datasets and even surpasses fully supervised results on KITTI 3D object detection benchmark.
  arXiv  Detail & Related papers  (2021-03-09T10:51:24Z)
• CorrNet3D: Unsupervised End-to-end Learning of Dense Correspondence for 3D Point Clouds [48.22275177437932]
  This paper addresses the problem of computing dense correspondence between 3D shapes in the form of point clouds. We propose CorrNet3D -- the first unsupervised and end-to-end deep learning-based framework.
  arXiv  Detail & Related papers  (2020-12-31T14:55:51Z)
• RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction [19.535169371240073]
  We introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from point clouds. We decouple the instance reconstruction into global object localization and local shape prediction. Our approach consistently outperforms the state-of-the-arts and improves over 11 of mesh IoU in object reconstruction.
  arXiv  Detail & Related papers  (2020-11-30T12:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.