Domain Adaptation of Synthetic Driving Datasets for Real-World Autonomous Driving

• URL: http://arxiv.org/abs/2302.04149v1
• Date: Wed, 8 Feb 2023 15:51:54 GMT
• Title: Domain Adaptation of Synthetic Driving Datasets for Real-World Autonomous Driving
• Authors: Koustav Mullick, Harshil Jain, Sanchit Gupta, Amit Arvind Kale
• Abstract summary: Network trained with synthetic data for certain computer vision tasks degrade significantly when tested on real world data. In this paper, we propose and evaluate novel ways for the betterment of such approaches. We propose a novel method to efficiently incorporate semantic supervision into this pair selection, which helps in boosting the performance of the model.
• Score: 0.11470070927586014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While developing perception based deep learning models, the benefit of synthetic data is enormous. However, performance of networks trained with synthetic data for certain computer vision tasks degrade significantly when tested on real world data due to the domain gap between them. One of the popular solutions in bridging this gap between synthetic and actual world data is to frame it as a domain adaptation task. In this paper, we propose and evaluate novel ways for the betterment of such approaches. In particular we build upon the method of UNIT-GAN. In normal GAN training for the task of domain translation, pairing of images from both the domains (viz, real and synthetic) is done randomly. We propose a novel method to efficiently incorporate semantic supervision into this pair selection, which helps in boosting the performance of the model along with improving the visual quality of such transformed images. We illustrate our empirical findings on Cityscapes \cite{cityscapes} and challenging synthetic dataset Synscapes. Though the findings are reported on the base network of UNIT-GAN, they can be easily extended to any other similar network.

Related papers

• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement [12.857137513211866]
  We propose an effective model training pipeline consisting of a training data synthesis and a gaze estimation model for unsupervised domain adaptation. The proposed data synthesis leverages the single-image 3D reconstruction to expand the range of the head poses from the source domain without requiring a 3D facial shape dataset. We propose a disentangling autoencoder network to separate gaze-related features and introduce background augmentation consistency loss to utilize the characteristics of the synthetic source domain.
  arXiv  Detail & Related papers  (2023-05-25T15:15:03Z)
• Bridging the Gap: Enhancing the Utility of Synthetic Data via Post-Processing Techniques [7.967995669387532]
  generative models have emerged as a promising solution for generating synthetic datasets that can replace or augment real-world data. We propose three novel post-processing techniques to improve the quality and diversity of the synthetic dataset. Experiments show that Gap Filler (GaFi) effectively reduces the gap with real-accuracy scores to an error of 2.03%, 1.78%, and 3.99% on the Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets, respectively.
  arXiv  Detail & Related papers  (2023-05-17T10:50:38Z)
• Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances [76.34037366117234]
  We introduce a new dataset called Robot Control Gestures (RoCoG-v2) The dataset is composed of both real and synthetic videos from seven gesture classes. We present results using state-of-the-art action recognition and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T23:23:55Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation [117.3856882511919]
  We propose the Style-HAllucinated Dual consistEncy learning (SHADE) framework to handle domain shift. Our SHADE yields significant improvement and outperforms state-of-the-art methods by 5.07% and 8.35% on the average mIoU of three real-world datasets.
  arXiv  Detail & Related papers  (2022-04-06T02:49:06Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Content Disentanglement for Semantically Consistent Synthetic-to-RealDomain Adaptation in Urban Traffic Scenes [39.38387505091648]
  Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. Deep learning techniques trained solely on synthetic data encounter dramatic performance drops when they are tested on real data. We propose a new, unsupervised, end-to-end domain adaptation network architecture that enables semantically consistent domain adaptation between synthetic and real data.
  arXiv  Detail & Related papers  (2021-05-18T17:42:26Z)
• Learning to Segment Human Body Parts with Synthetically Trained Deep Convolutional Networks [58.0240970093372]
  This paper presents a new framework for human body part segmentation based on Deep Convolutional Neural Networks trained using only synthetic data. The proposed approach achieves cutting-edge results without the need of training the models with real annotated data of human body parts.
  arXiv  Detail & Related papers  (2021-02-02T12:26:50Z)
• Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment [28.61782696432711]
  We propose a novel Unsupervised Domain Adaptation (UDA) strategy to address the domain shift issue between real world and synthetic representations. We show how the proposed strategy is able to obtain impressive performance in adapting a segmentation network trained on synthetic data to real world scenarios.
  arXiv  Detail & Related papers  (2020-01-14T10:12:20Z)
• Virtual to Real adaptation of Pedestrian Detectors [9.432150710329607]
  ViPeD is a new synthetically generated set of images collected with the graphical engine of the video game GTA V - Grand Theft Auto V. We propose two different Domain Adaptation techniques suitable for the pedestrian detection task, but possibly applicable to general object detection. Experiments show that the network trained with ViPeD can generalize over unseen real-world scenarios better than the detector trained over real-world data.
  arXiv  Detail & Related papers  (2020-01-09T14:50:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.