Style-transfer GANs for bridging the domain gap in synthetic pose estimator training

• URL: http://arxiv.org/abs/2004.13681v2
• Date: Wed, 16 Dec 2020 19:06:13 GMT
• Title: Style-transfer GANs for bridging the domain gap in synthetic pose estimator training
• Authors: Pavel Rojtberg, Thomas P\"ollabauer and Arjan Kuijper
• Abstract summary: We propose to adopt general-purpose GAN models for pixel-level image translation. The obtained models are then used either during training or inference to bridge the domain gap. Our evaluation shows a considerable improvement in model performance when compared to a model trained with the same degree of domain randomization.
• Score: 8.508403388002133
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Given the dependency of current CNN architectures on a large training set, the possibility of using synthetic data is alluring as it allows generating a virtually infinite amount of labeled training data. However, producing such data is a non-trivial task as current CNN architectures are sensitive to the domain gap between real and synthetic data. We propose to adopt general-purpose GAN models for pixel-level image translation, allowing to formulate the domain gap itself as a learning problem. The obtained models are then used either during training or inference to bridge the domain gap. Here, we focus on training the single-stage YOLO6D object pose estimator on synthetic CAD geometry only, where not even approximate surface information is available. When employing paired GAN models, we use an edge-based intermediate domain and introduce different mappings to represent the unknown surface properties. Our evaluation shows a considerable improvement in model performance when compared to a model trained with the same degree of domain randomization, while requiring only very little additional effort.

Related papers

• xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing [21.37585797507323]
  Cross-domain policy transfer methods mostly aim at learning domain correspondences or corrections to facilitate policy learning. We propose the Cross-Domain Trajectory EDiting framework that employs a specially designed diffusion model for cross-domain trajectory adaptation. Our proposed model architecture effectively captures the intricate dependencies among states, actions, and rewards, as well as the dynamics patterns within target data.
  arXiv  Detail & Related papers  (2024-09-13T10:07:28Z)
• Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation [18.011044932979143]
  3DUDA is a method capable of adapting to a nuisance-ridden target domain without 3D or depth data. We represent object categories as simple cuboid meshes, and harness a generative model of neural feature activations. We show that our method simulates fine-tuning on a global pseudo-labeled dataset under mild assumptions.
  arXiv  Detail & Related papers  (2024-01-19T17:48:05Z)
• Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation [65.78246406460305]
  compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
  arXiv  Detail & Related papers  (2023-08-28T14:43:36Z)
• Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement [12.857137513211866]
  We propose an effective model training pipeline consisting of a training data synthesis and a gaze estimation model for unsupervised domain adaptation. The proposed data synthesis leverages the single-image 3D reconstruction to expand the range of the head poses from the source domain without requiring a 3D facial shape dataset. We propose a disentangling autoencoder network to separate gaze-related features and introduce background augmentation consistency loss to utilize the characteristics of the synthetic source domain.
  arXiv  Detail & Related papers  (2023-05-25T15:15:03Z)
• Robust Category-Level 3D Pose Estimation from Synthetic Data [17.247607850702558]
  We introduce SyntheticP3D, a new synthetic dataset for object pose estimation generated from CAD models. We propose a novel approach (CC3D) for training neural mesh models that perform pose estimation via inverse rendering.
  arXiv  Detail & Related papers  (2023-05-25T14:56:03Z)
• HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments [0.0]
  This study uses domain randomization to generate a synthetic RGB-D dataset for training multimodal instance segmentation models. We show that our approach enables the models to outperform corresponding models trained on existing state-of-the-art datasets.
  arXiv  Detail & Related papers  (2023-04-12T13:02:08Z)
• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation [62.889835139583965]
  We introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.
  arXiv  Detail & Related papers  (2023-04-06T17:36:23Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• One-Shot Domain Adaptive and Generalizable Semantic Segmentation with Class-Aware Cross-Domain Transformers [96.51828911883456]
  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation aims to improve the real-world test performance of a model trained on simulated data. Traditional UDA often assumes that there are abundant unlabeled real-world data samples available during training for the adaptation. We explore the one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization problem, where only one real-world data sample is available.
  arXiv  Detail & Related papers  (2022-12-14T15:54:15Z)
• Adapting the Mean Teacher for keypoint-based lung registration under geometric domain shifts [75.51482952586773]
  deep neural networks generally require plenty of labeled training data and are vulnerable to domain shifts between training and test data. We present a novel approach to geometric domain adaptation for image registration, adapting a model from a labeled source to an unlabeled target domain. Our method consistently improves on the baseline model by 50%/47% while even matching the accuracy of models trained on target data.
  arXiv  Detail & Related papers  (2022-07-01T12:16:42Z)
• FedILC: Weighted Geometric Mean and Invariant Gradient Covariance for Federated Learning on Non-IID Data [69.0785021613868]
  Federated learning is a distributed machine learning approach which enables a shared server model to learn by aggregating the locally-computed parameter updates with the training data from spatially-distributed client silos. We propose the Federated Invariant Learning Consistency (FedILC) approach, which leverages the gradient covariance and the geometric mean of Hessians to capture both inter-silo and intra-silo consistencies. This is relevant to various fields such as medical healthcare, computer vision, and the Internet of Things (IoT)
  arXiv  Detail & Related papers  (2022-05-19T03:32:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.