Content Disentanglement for Semantically Consistent Synthetic-to-RealDomain Adaptation in Urban Traffic Scenes

• URL: http://arxiv.org/abs/2105.08704v1
• Date: Tue, 18 May 2021 17:42:26 GMT
• Title: Content Disentanglement for Semantically Consistent Synthetic-to-RealDomain Adaptation in Urban Traffic Scenes
• Authors: Mert Keser, Artem Savkin, Federico Tombari
• Abstract summary: Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. Deep learning techniques trained solely on synthetic data encounter dramatic performance drops when they are tested on real data. We propose a new, unsupervised, end-to-end domain adaptation network architecture that enables semantically consistent domain adaptation between synthetic and real data.
• Score: 39.38387505091648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. However, deep learning techniques trained solely on synthetic data encounter dramatic performance drops when they are tested on real data. Such performance drop is commonly attributed to the domain gap between real and synthetic data. Domain adaptation methods have been applied to mitigate the aforementioned domain gap. These methods achieve visually appealing results, but the translated samples usually introduce semantic inconsistencies. In this work, we propose a new, unsupervised, end-to-end domain adaptation network architecture that enables semantically consistent domain adaptation between synthetic and real data. We evaluate our architecture on the downstream task of semantic segmentation and show that our method achieves superior performance compared to the state-of-the-art methods.

Related papers

• StyDeSty: Min-Max Stylization and Destylization for Single Domain Generalization [85.18995948334592]
  Single domain generalization (single DG) aims at learning a robust model generalizable to unseen domains from only one training domain. State-of-the-art approaches have mostly relied on data augmentations, such as adversarial perturbation and style enhancement, to synthesize new data. We propose emphStyDeSty, which explicitly accounts for the alignment of the source and pseudo domains in the process of data augmentation.
  arXiv  Detail & Related papers  (2024-06-01T02:41:34Z)
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation [62.889835139583965]
  We introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.
  arXiv  Detail & Related papers  (2023-04-06T17:36:23Z)
• Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances [76.34037366117234]
  We introduce a new dataset called Robot Control Gestures (RoCoG-v2) The dataset is composed of both real and synthetic videos from seven gesture classes. We present results using state-of-the-art action recognition and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T23:23:55Z)
• Domain Adaptation of Synthetic Driving Datasets for Real-World Autonomous Driving [0.11470070927586014]
  Network trained with synthetic data for certain computer vision tasks degrade significantly when tested on real world data. In this paper, we propose and evaluate novel ways for the betterment of such approaches. We propose a novel method to efficiently incorporate semantic supervision into this pair selection, which helps in boosting the performance of the model.
  arXiv  Detail & Related papers  (2023-02-08T15:51:54Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision [73.76277367528657]
  Convolutional neural network-based approaches have achieved remarkable progress in semantic segmentation. To cope with this limitation, automatically annotated data generated from graphic engines are used to train segmentation models. We propose a two-step self-supervised domain adaptation approach to minimize the inter-domain and intra-domain gap together.
  arXiv  Detail & Related papers  (2020-04-16T15:24:11Z)
• Focus on Semantic Consistency for Cross-domain Crowd Understanding [34.560447389853614]
  Some domain adaptation algorithms try to liberate it by training models with synthetic data. We found that a mass of estimation errors in the background areas impede the performance of the existing methods. In this paper, we propose a domain adaptation method to eliminate it.
  arXiv  Detail & Related papers  (2020-02-20T08:51:05Z)
• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment [28.61782696432711]
  We propose a novel Unsupervised Domain Adaptation (UDA) strategy to address the domain shift issue between real world and synthetic representations. We show how the proposed strategy is able to obtain impressive performance in adapting a segmentation network trained on synthetic data to real world scenarios.
  arXiv  Detail & Related papers  (2020-01-14T10:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.