Semi-Supervised Image-to-Image Translation using Latent Space Mapping

• URL: http://arxiv.org/abs/2203.15241v1
• Date: Tue, 29 Mar 2022 05:14:26 GMT
• Title: Semi-Supervised Image-to-Image Translation using Latent Space Mapping
• Authors: Pan Zhang, Jianmin Bao, Ting Zhang, Dong Chen, Fang Wen
• Abstract summary: We introduce a general framework for semi-supervised image translation. Our main idea is to learn the translation over the latent feature space instead of the image space. Thanks to the low dimensional feature space, it is easier to find the desired mapping function.
• Score: 37.232496213047845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent image-to-image translation works have been transferred from supervised to unsupervised settings due to the expensive cost of capturing or labeling large amounts of paired data. However, current unsupervised methods using the cycle-consistency constraint may not find the desired mapping, especially for difficult translation tasks. On the other hand, a small number of paired data are usually accessible. We therefore introduce a general framework for semi-supervised image translation. Unlike previous works, our main idea is to learn the translation over the latent feature space instead of the image space. Thanks to the low dimensional feature space, it is easier to find the desired mapping function, resulting in improved quality of translation results as well as the stability of the translation model. Empirically we show that using feature translation generates better results, even using a few bits of paired data. Experimental comparisons with state-of-the-art approaches demonstrate the effectiveness of the proposed framework on a variety of challenging image-to-image translation tasks

Related papers

• Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applications [1.8011391924021904]
  This study empirically investigates unpaired image translation methods for generating suitable data in surgical applications. We find that a simple combination of structural-similarity loss and contrastive learning yields the most promising results.
  arXiv  Detail & Related papers  (2023-09-06T14:43:22Z)
• Unsupervised Image-to-Image Translation with Generative Prior [103.54337984566877]
  Unsupervised image-to-image translation aims to learn the translation between two visual domains without paired data. We present a novel framework, Generative Prior-guided UN Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
  arXiv  Detail & Related papers  (2022-04-07T17:59:23Z)
• Unbalanced Feature Transport for Exemplar-based Image Translation [51.54421432912801]
  This paper presents a general image translation framework that incorporates optimal transport for feature alignment between conditional inputs and style exemplars in image translation. We show that our method achieves superior image translation qualitatively and quantitatively as compared with the state-of-the-art.
  arXiv  Detail & Related papers  (2021-06-19T12:07:48Z)
• Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation [56.55178339375146]
  Image-to-Image (I2I) multi-domain translation models are usually evaluated also using the quality of their semantic results. We propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space.
  arXiv  Detail & Related papers  (2021-06-16T17:58:21Z)
• The Spatially-Correlative Loss for Various Image Translation Tasks [69.62228639870114]
  We propose a novel spatially-correlative loss that is simple, efficient and yet effective for preserving scene structure consistency. Previous methods attempt this by using pixel-level cycle-consistency or feature-level matching losses. We show distinct improvement over baseline models in all three modes of unpaired I2I translation: single-modal, multi-modal, and even single-image translation.
  arXiv  Detail & Related papers  (2021-04-02T02:13:30Z)
• Multiple GAN Inversion for Exemplar-based Image-to-Image Translation [0.0]
  We propose Multiple GAN Inversion for Exemplar-based Image-to-Image Translation. Our novel Multiple GAN Inversion avoids human intervention using a self-deciding algorithm in choosing the number of layers. Experimental results shows the advantage of the proposed method compared to existing state-of-the-art exemplar-based image-to-image translation methods.
  arXiv  Detail & Related papers  (2021-03-26T13:46:14Z)
• Rethinking the Truly Unsupervised Image-to-Image Translation [29.98784909971291]
  Unsupervised image-to-image translation model (TUNIT) learns to separate image domains and translates input images into estimated domains. Experimental results show TUNIT achieves comparable or even better performance than the set-level supervised model trained with full labels. TUNIT can be easily extended to semi-supervised learning with a few labeled data.
  arXiv  Detail & Related papers  (2020-06-11T15:15:12Z)
• Semi-supervised Learning for Few-shot Image-to-Image Translation [89.48165936436183]
  We propose a semi-supervised method for few-shot image translation, called SEMIT. Our method achieves excellent results on four different datasets using as little as 10% of the source labels.
  arXiv  Detail & Related papers  (2020-03-30T22:46:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.