DualCross: Cross-Modality Cross-Domain Adaptation for Monocular BEV Perception

• URL: http://arxiv.org/abs/2305.03724v2
• Date: Wed, 12 Jun 2024 00:35:15 GMT
• Title: DualCross: Cross-Modality Cross-Domain Adaptation for Monocular BEV Perception
• Authors: Yunze Man, Liang-Yan Gui, Yu-Xiong Wang,
• Abstract summary: DualCross is a cross-modality cross-domain adaptation framework to facilitate the learning of a more robust BEV perception model. This work results in the first open analysis of cross-domain cross-sensor perception and adaptation for monocular 3D tasks in the wild.
• Score: 30.113617846516398
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Closing the domain gap between training and deployment and incorporating multiple sensor modalities are two challenging yet critical topics for self-driving. Existing work only focuses on single one of the above topics, overlooking the simultaneous domain and modality shift which pervasively exists in real-world scenarios. A model trained with multi-sensor data collected in Europe may need to run in Asia with a subset of input sensors available. In this work, we propose DualCross, a cross-modality cross-domain adaptation framework to facilitate the learning of a more robust monocular bird's-eye-view (BEV) perception model, which transfers the point cloud knowledge from a LiDAR sensor in one domain during the training phase to the camera-only testing scenario in a different domain. This work results in the first open analysis of cross-domain cross-sensor perception and adaptation for monocular 3D tasks in the wild. We benchmark our approach on large-scale datasets under a wide range of domain shifts and show state-of-the-art results against various baselines.

Related papers

• One for All: Multi-Domain Joint Training for Point Cloud Based 3D Object Detection [71.78795573911512]
  We propose textbfOneDet3D, a universal one-for-all model that addresses 3D detection across different domains. We propose the domain-aware in scatter and context, guided by a routing mechanism, to address the data interference issue. The fully sparse structure and anchor-free head further accommodate point clouds with significant scale disparities.
  arXiv  Detail & Related papers  (2024-11-03T14:21:56Z)
• M3BAT: Unsupervised Domain Adaptation for Multimodal Mobile Sensing with Multi-Branch Adversarial Training [5.128670847334003]
  multimodal mobile sensing has been used extensively for inferences regarding health and well being, behavior, and context. The distribution of data in the training set differs from the distribution of data in the real world, the deployment environment. We propose M3BAT, an unsupervised domain adaptation for multimodal mobile sensing with multi-branch adversarial training.
  arXiv  Detail & Related papers  (2024-04-26T13:09:35Z)
• Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation [65.78246406460305]
  compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
  arXiv  Detail & Related papers  (2023-08-28T14:43:36Z)
• BEV-DG: Cross-Modal Learning under Bird's-Eye View for Domain Generalization of 3D Semantic Segmentation [59.99683295806698]
  Cross-modal Unsupervised Domain Adaptation (UDA) aims to exploit the complementarity of 2D-3D data to overcome the lack of annotation in a new domain. We propose cross-modal learning under bird's-eye view for Domain Generalization (DG) of 3D semantic segmentation, called BEV-DG.
  arXiv  Detail & Related papers  (2023-08-12T11:09:17Z)
• Cross-Modality Domain Adaptation for Freespace Detection: A Simple yet Effective Baseline [21.197212665408262]
  Freespace detection aims at classifying each pixel of the image captured by the camera as drivable or non-drivable. We develop a cross-modality domain adaptation framework which exploits both RGB images and surface normal maps generated from depth images. To better bridge the domain gap between source domain (synthetic data) and target domain (real-world data), we also propose a Selective Feature Alignment (SFA) module.
  arXiv  Detail & Related papers  (2022-10-06T15:31:49Z)
• Dual Adversarial Adaptation for Cross-Device Real-World Image Super-Resolution [114.26933742226115]
  Super-resolution (SR) models trained on images from different devices could exhibit distinct imaging patterns. We propose an unsupervised domain adaptation mechanism for real-world SR, named Dual ADversarial Adaptation (DADA) We empirically conduct experiments under six Real to Real adaptation settings among three different cameras, and achieve superior performance compared with existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-05-07T02:55:39Z)
• An Unsupervised Domain Adaptive Approach for Multimodal 2D Object Detection in Adverse Weather Conditions [5.217255784808035]
  We propose an unsupervised domain adaptation framework to bridge the domain gap between source and target domains. We use a data augmentation scheme that simulates weather distortions to add domain confusion and prevent overfitting on the source data. Experiments performed on the DENSE dataset show that our method can substantially alleviate the domain gap.
  arXiv  Detail & Related papers  (2022-03-07T18:10:40Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency [90.71745178767203]
  Deep learning-based 3D object detection has achieved unprecedented success with the advent of large-scale autonomous driving datasets. Existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world. We study a more realistic setting, unsupervised 3D domain adaptive detection, which only utilizes source domain annotations.
  arXiv  Detail & Related papers  (2021-07-23T17:19:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.