UTSGAN: Unseen Transition Suss GAN for Transition-Aware Image-to-image Translation

• URL: http://arxiv.org/abs/2304.11955v1
• Date: Mon, 24 Apr 2023 09:47:34 GMT
• Title: UTSGAN: Unseen Transition Suss GAN for Transition-Aware Image-to-image Translation
• Authors: Yaxin Shi, Xiaowei Zhou, Ping Liu, Ivor W. Tsang
• Abstract summary: We introduce a transition-aware approach to I2I translation, where the data translation mapping is explicitly parameterized with a transition variable. We propose the use of transition consistency, defined on the transition variable, to enable regularization of consistency on unobserved translations. Based on these insights, we present Unseen Transition Suss GAN (UTSGAN), a generative framework that constructs a manifold for the transition with a transition encoder.
• Score: 57.99923293611923
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In the field of Image-to-Image (I2I) translation, ensuring consistency between input images and their translated results is a key requirement for producing high-quality and desirable outputs. Previous I2I methods have relied on result consistency, which enforces consistency between the translated results and the ground truth output, to achieve this goal. However, result consistency is limited in its ability to handle complex and unseen attribute changes in translation tasks. To address this issue, we introduce a transition-aware approach to I2I translation, where the data translation mapping is explicitly parameterized with a transition variable, allowing for the modelling of unobserved translations triggered by unseen transitions. Furthermore, we propose the use of transition consistency, defined on the transition variable, to enable regularization of consistency on unobserved translations, which is omitted in previous works. Based on these insights, we present Unseen Transition Suss GAN (UTSGAN), a generative framework that constructs a manifold for the transition with a stochastic transition encoder and coherently regularizes and generalizes result consistency and transition consistency on both training and unobserved translations with tailor-designed constraints. Extensive experiments on four different I2I tasks performed on five different datasets demonstrate the efficacy of our proposed UTSGAN in performing consistent translations.

Related papers

• ParGAN: Learning Real Parametrizable Transformations [50.51405390150066]
  We propose ParGAN, a generalization of the cycle-consistent GAN framework to learn image transformations. The proposed generator takes as input both an image and a parametrization of the transformation. We show how, with disjoint image domains with no annotated parametrization, our framework can create smooths as well as learn multiple transformations simultaneously.
  arXiv  Detail & Related papers  (2022-11-09T16:16:06Z)
• Beyond Deterministic Translation for Unsupervised Domain Adaptation [19.358300726820943]
  In this work we challenge the common approach of using a one-to-one mapping (''translation'') between the source and target domains in unsupervised domain adaptation (UDA) Instead, we rely on translation to capture inherent ambiguities between the source and target domains. We report improvements over strong recent baselines, leading to state-of-the-art UDA results on two challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-02-15T23:03:33Z)
• Unbalanced Feature Transport for Exemplar-based Image Translation [51.54421432912801]
  This paper presents a general image translation framework that incorporates optimal transport for feature alignment between conditional inputs and style exemplars in image translation. We show that our method achieves superior image translation qualitatively and quantitatively as compared with the state-of-the-art.
  arXiv  Detail & Related papers  (2021-06-19T12:07:48Z)
• Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation [56.55178339375146]
  Image-to-Image (I2I) multi-domain translation models are usually evaluated also using the quality of their semantic results. We propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space.
  arXiv  Detail & Related papers  (2021-06-16T17:58:21Z)
• Generative Transition Mechanism to Image-to-Image Translation via Encoded Transformation [40.11493448767101]
  We revisit the Image-to-Image (I2I) translation problem with transition consistency. Existing I2I translation models mainly focus on maintaining consistency on results. We propose to enforce both result consistency and transition consistency for I2I translation.
  arXiv  Detail & Related papers  (2021-03-09T02:56:03Z)
• Sign Language Transformers: Joint End-to-end Sign Language Recognition and Translation [59.38247587308604]
  We introduce a novel transformer based architecture that jointly learns Continuous Sign Language Recognition and Translation. We evaluate the recognition and translation performances of our approaches on the challenging RWTH-PHOENIX-Weather-2014T dataset. Our translation networks outperform both sign video to spoken language and gloss to spoken language translation models.
  arXiv  Detail & Related papers  (2020-03-30T21:35:09Z)
• Augmented Cyclic Consistency Regularization for Unpaired Image-to-Image Translation [22.51574923085135]
  Augmented Cyclic Consistency Regularization (A CCR) is a novel regularization method for unpaired I2I translation. Our method outperforms the consistency regularized GAN (CR-GAN) in real-world translations.
  arXiv  Detail & Related papers  (2020-02-29T06:20:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.