ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework
for LiDAR Point Cloud Segmentation
- URL: http://arxiv.org/abs/2009.03456v2
- Date: Tue, 23 Feb 2021 16:52:41 GMT
- Title: ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework
for LiDAR Point Cloud Segmentation
- Authors: Sicheng Zhao, Yezhen Wang, Bo Li, Bichen Wu, Yang Gao, Pengfei Xu,
Trevor Darrell, Kurt Keutzer
- Abstract summary: Training deep neural networks (DNNs) on LiDAR data requires large-scale point-wise annotations.
Simulation-to-real domain adaptation (SRDA) trains a DNN using unlimited synthetic data with automatically generated labels.
ePointDA consists of three modules: self-supervised dropout noise rendering, statistics-invariant and spatially-adaptive feature alignment, and transferable segmentation learning.
- Score: 111.56730703473411
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Due to its robust and precise distance measurements, LiDAR plays an important
role in scene understanding for autonomous driving. Training deep neural
networks (DNNs) on LiDAR data requires large-scale point-wise annotations,
which are time-consuming and expensive to obtain. Instead, simulation-to-real
domain adaptation (SRDA) trains a DNN using unlimited synthetic data with
automatically generated labels and transfers the learned model to real
scenarios. Existing SRDA methods for LiDAR point cloud segmentation mainly
employ a multi-stage pipeline and focus on feature-level alignment. They
require prior knowledge of real-world statistics and ignore the pixel-level
dropout noise gap and the spatial feature gap between different domains. In
this paper, we propose a novel end-to-end framework, named ePointDA, to address
the above issues. Specifically, ePointDA consists of three modules:
self-supervised dropout noise rendering, statistics-invariant and
spatially-adaptive feature alignment, and transferable segmentation learning.
The joint optimization enables ePointDA to bridge the domain shift at the
pixel-level by explicitly rendering dropout noise for synthetic LiDAR and at
the feature-level by spatially aligning the features between different domains,
without requiring the real-world statistics. Extensive experiments adapting
from synthetic GTA-LiDAR to real KITTI and SemanticKITTI demonstrate the
superiority of ePointDA for LiDAR point cloud segmentation.
Related papers
- PU-Ray: Domain-Independent Point Cloud Upsampling via Ray Marching on Neural Implicit Surface [5.78575346449322]
We propose a new ray-based upsampling approach with an arbitrary rate, where a depth prediction is made for each query ray and its corresponding patch.
Our novel method simulates the sphere-tracing ray marching algorithm on the neural implicit surface defined with an unsigned distance function (UDF)
The rule-based mid-point query sampling method generates more evenly distributed points without requiring an end-to-end model trained using a nearest-neighbor-based reconstruction loss function.
arXiv Detail & Related papers (2023-10-12T22:45:03Z) - Compositional Semantic Mix for Domain Adaptation in Point Cloud
Segmentation [65.78246406460305]
compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation.
We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
arXiv Detail & Related papers (2023-08-28T14:43:36Z) - One-Shot Domain Adaptive and Generalizable Semantic Segmentation with
Class-Aware Cross-Domain Transformers [96.51828911883456]
Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data.
Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation.
We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
arXiv Detail & Related papers (2022-12-14T15:54:15Z) - SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from
Point Cloud [125.9472454212909]
We present a novel Semi-Supervised Domain Adaptation method for 3D object detection (SSDA3D)
SSDA3D includes an Inter-domain Adaptation stage and an Intra-domain Generalization stage.
Experiments show that, with only 10% labeled target data, our SSDA3D can surpass the fully-supervised oracle model with 100% target label.
arXiv Detail & Related papers (2022-12-06T09:32:44Z) - Learning to Simulate Realistic LiDARs [66.7519667383175]
We introduce a pipeline for data-driven simulation of a realistic LiDAR sensor.
We show that our model can learn to encode realistic effects such as dropped points on transparent surfaces.
We use our technique to learn models of two distinct LiDAR sensors and use them to improve simulated LiDAR data accordingly.
arXiv Detail & Related papers (2022-09-22T13:12:54Z) - GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D
LiDAR Segmentation [60.07812405063708]
3D point cloud semantic segmentation is fundamental for autonomous driving.
Most approaches in the literature neglect an important aspect, i.e., how to deal with domain shift when handling dynamic scenes.
This paper advances the state of the art in this research field.
arXiv Detail & Related papers (2022-07-20T09:06:07Z) - Sim-to-Real Domain Adaptation for Lane Detection and Classification in
Autonomous Driving [0.0]
Unsupervised Domain Adaptation (UDA) approaches are considered low-cost and less time-consuming.
We propose UDA schemes using adversarial discriminative and generative methods for lane detection and classification applications in autonomous driving.
arXiv Detail & Related papers (2022-02-15T02:10:14Z) - Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation [5.745037250837124]
Unsupervised Domain Adaptation (UDA) techniques are essential to fill this domain gap.
We propose AdaptLPS, a novel UDA approach for LiDAR panoptic segmentation.
We show that AdaptLPS outperforms existing UDA approaches by up to 6.41 pp in terms of the PQ score.
arXiv Detail & Related papers (2021-09-30T17:30:43Z) - MNEW: Multi-domain Neighborhood Embedding and Weighting for Sparse Point
Clouds Segmentation [1.2380933178502298]
We propose MNEW, including multi-domain neighborhood embedding, and attention weighting based on their geometry distance, feature similarity, and neighborhood sparsity.
MNEW achieves the top performance for sparse point clouds, which is important to the application of LiDAR-based automated driving perception.
arXiv Detail & Related papers (2020-04-05T18:02:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.