CTS: Sim-to-Real Unsupervised Domain Adaptation on 3D Detection

• URL: http://arxiv.org/abs/2406.18129v2
• Date: Mon, 30 Sep 2024 07:07:31 GMT
• Title: CTS: Sim-to-Real Unsupervised Domain Adaptation on 3D Detection
• Authors: Meiying Zhang, Weiyuan Peng, Guangyao Ding, Chenyang Lei, Chunlin Ji, Qi Hao,
• Abstract summary: This paper presents a novel framework to transfer models from labeled simulation to unlabeled reality domains. Experimental results demonstrate that our proposed approach significantly enhances the sim-to-real domain adaptation capability of 3D object detection models.
• Score: 16.96201890965781
• License:
• Abstract: Simulation data can be accurately labeled and have been expected to improve the performance of data-driven algorithms, including object detection. However, due to the various domain inconsistencies from simulation to reality (sim-to-real),cross-domain object detection algorithms usually suffer from dramatic performance drops. While numerous unsupervised domain adaptation (UDA) methods have been developed to address cross-domain tasks between real-world datasets, progress in sim-to-real remains limited. This paper presents a novel Complex-to-Simple (CTS) framework to transfer models from labeled simulation (source) to unlabeled reality (target) domains. Based on a two-stage detector, the novelty of this work is threefold: 1) developing fixed-size anchor heads and RoI augmentation to address size bias and feature diversity between two domains, thereby improving the quality of pseudo-label; 2) developing a novel corner-format representation of aleatoric uncertainty (AU) for the bounding box, to uniformly quantify pseudo-label quality; 3) developing a noise-aware mean teacher domain adaptation method based on AU, as well as object-level and frame-level sampling strategies, to migrate the impact of noisy labels. Experimental results demonstrate that our proposed approach significantly enhances the sim-to-real domain adaptation capability of 3D object detection models, outperforming state-of-the-art cross-domain algorithms, which are usually developed for real-to-real UDA tasks.

Related papers

• STAL3D: Unsupervised Domain Adaptation for 3D Object Detection via Collaborating Self-Training and Adversarial Learning [21.063779140059157]
  Existing 3D object detection suffers from expensive annotation costs and poor transferability to unknown data due to the domain gap. We propose a novel unsupervised domain adaptation framework for 3D object detection via collaborating ST and AL, dubbed as STAL3D, unleashing the complementary advantages of pseudo labels and feature distribution alignment.
  arXiv  Detail & Related papers  (2024-06-27T17:43:35Z)
• Exploring Test-Time Adaptation for Object Detection in Continually Changing Environments [13.163784646113214]
  Continual Test-Time Adaptation (CTTA) has recently emerged as a promising technique to gradually adapt a source-trained model to continually changing target domains. We present AMROD, featuring three core components. Firstly, the object-level contrastive learning module extracts object-level features for contrastive learning to refine the feature representation in the target domain. Secondly, the adaptive monitoring module dynamically skips unnecessary adaptation and updates the category-specific threshold based on predicted confidence scores to enable efficiency and improve the quality of pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-24T08:30:03Z)
• Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation [18.011044932979143]
  3DUDA is a method capable of adapting to a nuisance-ridden target domain without 3D or depth data. We represent object categories as simple cuboid meshes, and harness a generative model of neural feature activations. We show that our method simulates fine-tuning on a global pseudo-labeled dataset under mild assumptions.
  arXiv  Detail & Related papers  (2024-01-19T17:48:05Z)
• Hierarchical Disentanglement-Alignment Network for Robust SAR Vehicle Recognition [18.38295403066007]
  HDANet integrates feature disentanglement and alignment into a unified framework. The proposed method demonstrates impressive robustness across nine operating conditions in the MSTAR dataset.
  arXiv  Detail & Related papers  (2023-04-07T09:11:29Z)
• Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting [72.75792823726479]
  We propose Conditional Domain Translation via Conditional Alignment and Reweighting (CARE) to close the sim2real appearance and content gaps. We present an analytical justification of our algorithm and demonstrate strong gains over competing methods on standard benchmarks.
  arXiv  Detail & Related papers  (2023-02-09T18:39:28Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Unsupervised Domain Adaptation for Monocular 3D Object Detection via Self-Training [57.25828870799331]
  We propose STMono3D, a new self-teaching framework for unsupervised domain adaptation on Mono3D. We develop a teacher-student paradigm to generate adaptive pseudo labels on the target domain. STMono3D achieves remarkable performance on all evaluated datasets and even surpasses fully supervised results on the KITTI 3D object detection dataset.
  arXiv  Detail & Related papers  (2022-04-25T12:23:07Z)
• Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency [90.71745178767203]
  Deep learning-based 3D object detection has achieved unprecedented success with the advent of large-scale autonomous driving datasets. Existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world. We study a more realistic setting, unsupervised 3D domain adaptive detection, which only utilizes source domain annotations.
  arXiv  Detail & Related papers  (2021-07-23T17:19:23Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)
• Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View [110.83289076967895]
  We present a BEV domain adaptation method based on CycleGAN that uses prior semantic classification in order to preserve the information of small objects of interest during the domain adaptation process. The quality of the generated BEVs has been evaluated using a state-of-the-art 3D object detection framework at KITTI 3D Object Detection Benchmark.
  arXiv  Detail & Related papers  (2021-04-22T12:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.