ReSimAD: Zero-Shot 3D Domain Transfer for Autonomous Driving with Source Reconstruction and Target Simulation

• URL: http://arxiv.org/abs/2309.05527v4
• Date: Thu, 25 Jan 2024 10:51:45 GMT
• Title: ReSimAD: Zero-Shot 3D Domain Transfer for Autonomous Driving with Source Reconstruction and Target Simulation
• Abstract summary: We provide a new perspective and approach of alleviating the domain shifts, by proposing a Reconstruction-Simulation-Perception scheme. Specifically, the implicit reconstruction process is based on the knowledge from the previous old domain, aiming to convert the domain-related knowledge into domain-invariant representations. Our method is beneficial to boost the domain ability, even promising for 3D pre-training.
• Score: 59.268764137560524
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain shifts such as sensor type changes and geographical situation variations are prevalent in Autonomous Driving (AD), which poses a challenge since AD model relying on the previous domain knowledge can be hardly directly deployed to a new domain without additional costs. In this paper, we provide a new perspective and approach of alleviating the domain shifts, by proposing a Reconstruction-Simulation-Perception (ReSimAD) scheme. Specifically, the implicit reconstruction process is based on the knowledge from the previous old domain, aiming to convert the domain-related knowledge into domain-invariant representations, e.g., 3D scene-level meshes. Besides, the point clouds simulation process of multiple new domains is conditioned on the above reconstructed 3D meshes, where the target-domain-like simulation samples can be obtained, thus reducing the cost of collecting and annotating new-domain data for the subsequent perception process. For experiments, we consider different cross-domain situations such as Waymo-to-KITTI, Waymo-to-nuScenes, Waymo-to-ONCE, etc, to verify the zero-shot target-domain perception using ReSimAD. Results demonstrate that our method is beneficial to boost the domain generalization ability, even promising for 3D pre-training.

Related papers

• Parameter isolation with domain-specific experts for incremental audio classification [4.541172620857825]
  This paper proposes a new domain-specific parameter-isolation architecture that retains all past domains.<n>We incorporate data-free generative replay to reconstruct previous-domain data and cross-domain feature generation to recover later expert features missing from earlier domains.<n>We achieve micro and macro accuracies of 78.4% and 78.9%, respectively, representing increases of 33 and 25 percentage points over the Challenge baseline.
  arXiv  Detail & Related papers  (2026-09-13T18:47:01Z)
• DOME: Learning Transferable Domain Variables from Sparse Supervision for Test-Time Adaptation [61.70497741321102]
  Test-time adaptation aims to align a model to shifting test domains using only unlabeled streaming data.<n>Most existing methods implicitly infer a single global domain distribution, ignoring the multidimensional and sample-specific nature of real-world domain shifts.<n>We propose DOME, an effective domain encoder that explicitly models each sample's domain in a zero-shot manner.
  arXiv  Detail & Related papers  (2026-06-02T03:24:03Z)
• Adapting to Distribution Shift by Visual Domain Prompt Generation [34.19066857066073]
  We adapt a model at test-time using a few unlabeled data to address distribution shifts. We build a knowledge bank to learn the transferable knowledge from source domains. The proposed method outperforms previous work on 5 large-scale benchmarks including WILDS and DomainNet.
  arXiv  Detail & Related papers  (2024-05-05T02:44:04Z)
• Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer [69.82229895838577]
  Active Domain Adaptation (ADA) aims to maximally boost model adaptation in a new target domain by actively selecting a limited number of target data to annotate. This setting neglects the more practical scenario where training data are collected from multiple sources. This motivates us to target a new and challenging setting of knowledge transfer that extends ADA from a single source domain to multiple source domains.
  arXiv  Detail & Related papers  (2023-11-21T13:12:21Z)
• Meta-causal Learning for Single Domain Generalization [102.53303707563612]
  Single domain generalization aims to learn a model from a single training domain (source domain) and apply it to multiple unseen test domains (target domains) Existing methods focus on expanding the distribution of the training domain to cover the target domains, but without estimating the domain shift between the source and target domains. We propose a new learning paradigm, namely simulate-analyze-reduce, which first simulates the domain shift by building an auxiliary domain as the target domain, then learns to analyze the causes of domain shift, and finally learns to reduce the domain shift for model adaptation.
  arXiv  Detail & Related papers  (2023-04-07T15:46:38Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• Domain Re-Modulation for Few-Shot Generative Domain Adaptation [71.47730150327818]
  Generative Domain Adaptation (GDA) involves transferring a pre-trained generator from one domain to a new domain using only a few reference images. Inspired by the way human brains acquire knowledge in new domains, we present an innovative generator structure called Domain Re-Modulation (DoRM) DoRM not only meets the criteria of high quality, large synthesis diversity, and cross-domain consistency, but also incorporates memory and domain association.
  arXiv  Detail & Related papers  (2023-02-06T03:55:35Z)
• Foresee What You Will Learn: Data Augmentation for Domain Generalization in Non-Stationary Environments [14.344721944207599]
  Existing domain generalization aims to learn a generalizable model to perform well even on unseen domains. We propose Directional Domain Augmentation (DDA), which simulates the unseen target features by mapping source data as augmentations through a domain transformer. We evaluate the proposed method on both synthetic datasets and realworld datasets, and empirical results show that our approach can outperform other existing methods.
  arXiv  Detail & Related papers  (2023-01-19T01:51:37Z)
• Adversarial Bi-Regressor Network for Domain Adaptive Regression [52.5168835502987]
  It is essential to learn a cross-domain regressor to mitigate the domain shift. This paper proposes a novel method Adversarial Bi-Regressor Network (ABRNet) to seek more effective cross-domain regression model.
  arXiv  Detail & Related papers  (2022-09-20T18:38:28Z)
• Variational Transfer Learning using Cross-Domain Latent Modulation [1.9662978733004601]
  We introduce a novel cross-domain latent modulation mechanism to a variational autoencoder framework so as to achieve effective transfer learning. Deep representations of the source and target domains are first extracted by a unified inference model and aligned by employing gradient reversal. The learned deep representations are then cross-modulated to the latent encoding of the alternative domain, where consistency constraints are also applied.
  arXiv  Detail & Related papers  (2022-05-31T03:47:08Z)
• Domain Adaptation for Real-World Single View 3D Reconstruction [1.611271868398988]
  unsupervised domain adaptation can be used to transfer knowledge from the labeled synthetic source domain to the unlabeled real target domain. We propose a novel architecture which takes advantage of the fact that in this setting, target domain data is unsupervised with regards to the 3D model but supervised for class labels. Results are performed with ShapeNet as the source domain and domains within the Object Domain Suite (ODDS) dataset as the target.
  arXiv  Detail & Related papers  (2021-08-24T22:02:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.