Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models

• URL: http://arxiv.org/abs/2411.14495v1
• Date: Thu, 21 Nov 2024 00:04:38 GMT
• Title: Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models
• Authors: Hamidreza Dastmalchi, Aijun An, Ali Cheraghian, Shafin Rahman, Sameera Ramasinghe,
• Abstract summary: Test-time adaptation of 3D point clouds is crucial for mitigating discrepancies between training and testing samples in real-world scenarios. We introduce a novel 3D test-time adaptation method, termed 3DD-TTA, which stands for 3D Denoising Diffusion Test-Time Adaptation.
• Score: 19.795578581043745
• License:
• Abstract: Test-time adaptation (TTA) of 3D point clouds is crucial for mitigating discrepancies between training and testing samples in real-world scenarios, particularly when handling corrupted point clouds. LiDAR data, for instance, can be affected by sensor failures or environmental factors, causing domain gaps. Adapting models to these distribution shifts online is crucial, as training for every possible variation is impractical. Existing methods often focus on fine-tuning pre-trained models based on self-supervised learning or pseudo-labeling, which can lead to forgetting valuable source domain knowledge over time and reduce generalization on future tests. In this paper, we introduce a novel 3D test-time adaptation method, termed 3DD-TTA, which stands for 3D Denoising Diffusion Test-Time Adaptation. This method uses a diffusion strategy that adapts input point cloud samples to the source domain while keeping the source model parameters intact. The approach uses a Variational Autoencoder (VAE) to encode the corrupted point cloud into a shape latent and latent points. These latent points are corrupted with Gaussian noise and subjected to a denoising diffusion process. During this process, both the shape latent and latent points are updated to preserve fidelity, guiding the denoising toward generating consistent samples that align more closely with the source domain. We conduct extensive experiments on the ShapeNet dataset and investigate its generalizability on ModelNet40 and ScanObjectNN, achieving state-of-the-art results. The code has been released at \url{https://github.com/hamidreza-dastmalchi/3DD-TTA}.

Related papers

• Test-Time Adaptation in Point Clouds: Leveraging Sampling Variation with Weight Averaging [17.74824534094739]
  Test-Time Adaptation (TTA) addresses distribution shifts during testing by adapting a pretrained model without access to source data. We propose a novel TTA approach for 3D point cloud classification, combining sampling variation with weight averaging.
  arXiv  Detail & Related papers  (2024-11-02T02:59:25Z)
• Diff3DETR:Agent-based Diffusion Model for Semi-supervised 3D Object Detection [33.58208166717537]
  3D object detection is essential for understanding 3D scenes. Recent developments in semi-supervised methods seek to mitigate this problem by employing a teacher-student framework to generate pseudo-labels for unlabeled point clouds. We introduce an Agent-based Diffusion Model for Semi-supervised 3D Object Detection (Diff3DETR)
  arXiv  Detail & Related papers  (2024-08-01T05:04:22Z)
• CloudFixer: Test-Time Adaptation for 3D Point Clouds via Diffusion-Guided Geometric Transformation [33.07886526437753]
  3D point clouds captured from real-world sensors frequently encompass noisy points due to various obstacles. These challenges hinder the deployment of pre-trained point cloud recognition models trained on clean point clouds. We propose CloudFixer, a test-time input adaptation method tailored for 3D point clouds.
  arXiv  Detail & Related papers  (2024-07-23T05:35:04Z)
• Addressing Concept Shift in Online Time Series Forecasting: Detect-then-Adapt [37.98336090671441]
  Concept textbfDrift textbfDetection antextbfD textbfAdaptation (D3A) It first detects drifting conception and then aggressively adapts the current model to the drifted concepts after the detection for rapid adaption. It helps mitigate the data distribution gap, a critical factor contributing to train-test performance inconsistency.
  arXiv  Detail & Related papers  (2024-03-22T04:44:43Z)
• Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network [22.81108868492533]
  We propose a 3D anomaly synthesis pipeline to adapt existing large-scale 3Dmodels for 3D anomaly detection. Anomaly-ShapeNet consists of 1600 point cloud samples under 40 categories, which provides a rich and varied collection of data. We also propose a self-supervised method, i.e., Iterative Mask Reconstruction Network (IMRNet), to enable scalable representation learning for 3D anomaly localization.
  arXiv  Detail & Related papers  (2023-11-25T01:45:09Z)
• Diffusion-based 3D Object Detection with Random Boxes [58.43022365393569]
  Existing anchor-based 3D detection methods rely on empiricals setting of anchors, which makes the algorithms lack elegance. Our proposed Diff3Det migrates the diffusion model to proposal generation for 3D object detection by considering the detection boxes as generative targets. In the inference stage, the model progressively refines a set of random boxes to the prediction results.
  arXiv  Detail & Related papers  (2023-09-05T08:49:53Z)
• 3D Adversarial Augmentations for Robust Out-of-Domain Predictions [115.74319739738571]
  We focus on improving the generalization to out-of-domain data. We learn a set of vectors that deform the objects in an adversarial fashion. We perform adversarial augmentation by applying the learned sample-independent vectors to the available objects when training a model.
  arXiv  Detail & Related papers  (2023-08-29T17:58:55Z)
• StarNet: Style-Aware 3D Point Cloud Generation [82.30389817015877]
  StarNet is able to reconstruct and generate high-fidelity and even 3D point clouds using a mapping network. Our framework achieves comparable state-of-the-art performance on various metrics in the point cloud reconstruction and generation tasks.
  arXiv  Detail & Related papers  (2023-03-28T08:21:44Z)
• SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloud [125.9472454212909]
  We present a novel Semi-Supervised Domain Adaptation method for 3D object detection (SSDA3D) SSDA3D includes an Inter-domain Adaptation stage and an Intra-domain Generalization stage. Experiments show that, with only 10% labeled target data, our SSDA3D can surpass the fully-supervised oracle model with 100% target label.
  arXiv  Detail & Related papers  (2022-12-06T09:32:44Z)
• Back to the Source: Diffusion-Driven Test-Time Adaptation [77.4229736436935]
  Test-time adaptation harnesses test inputs to improve accuracy of a model trained on source data when tested on shifted target data. We instead update the target data, by projecting all test inputs toward the source domain with a generative diffusion model.
  arXiv  Detail & Related papers  (2022-07-07T17:14:10Z)
• Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection [85.11649974840758]
  3D object detection networks tend to be biased towards the data they are trained on. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors.
  arXiv  Detail & Related papers  (2021-11-30T18:42:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.