Retrieval Guided Unsupervised Multi-domain Image-to-Image Translation

• URL: http://arxiv.org/abs/2008.04991v1
• Date: Tue, 11 Aug 2020 20:11:53 GMT
• Title: Retrieval Guided Unsupervised Multi-domain Image-to-Image Translation
• Authors: Raul Gomez, Yahui Liu, Marco De Nadai, Dimosthenis Karatzas, Bruno Lepri and Nicu Sebe
• Abstract summary: Image to image translation aims to learn a mapping that transforms an image from one visual domain to another. We propose the use of an image retrieval system to assist the image-to-image translation task.
• Score: 59.73535607392732
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image to image translation aims to learn a mapping that transforms an image from one visual domain to another. Recent works assume that images descriptors can be disentangled into a domain-invariant content representation and a domain-specific style representation. Thus, translation models seek to preserve the content of source images while changing the style to a target visual domain. However, synthesizing new images is extremely challenging especially in multi-domain translations, as the network has to compose content and style to generate reliable and diverse images in multiple domains. In this paper we propose the use of an image retrieval system to assist the image-to-image translation task. First, we train an image-to-image translation model to map images to multiple domains. Then, we train an image retrieval model using real and generated images to find images similar to a query one in content but in a different domain. Finally, we exploit the image retrieval system to fine-tune the image-to-image translation model and generate higher quality images. Our experiments show the effectiveness of the proposed solution and highlight the contribution of the retrieval network, which can benefit from additional unlabeled data and help image-to-image translation models in the presence of scarce data.

Related papers

• Conditional Diffusion on Web-Scale Image Pairs leads to Diverse Image Variations [32.892042877725125]
  Current image variation techniques involve adapting a text-to-image model to reconstruct an input image conditioned on the same image. We show that a diffusion model trained to reconstruct an input image from frozen embeddings, can reconstruct the image with minor variations. We propose a new pretraining strategy to generate image variations using a large collection of image pairs.
  arXiv  Detail & Related papers  (2024-05-23T17:58:03Z)
• SCONE-GAN: Semantic Contrastive learning-based Generative Adversarial Network for an end-to-end image translation [18.93434486338439]
  SCONE-GAN is shown to be effective for learning to generate realistic and diverse scenery images. For more realistic and diverse image generation we introduce style reference image. We validate the proposed algorithm for image-to-image translation and stylizing outdoor images.
  arXiv  Detail & Related papers  (2023-11-07T10:29:16Z)
• Unsupervised Image-to-Image Translation with Generative Prior [103.54337984566877]
  Unsupervised image-to-image translation aims to learn the translation between two visual domains without paired data. We present a novel framework, Generative Prior-guided UN Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
  arXiv  Detail & Related papers  (2022-04-07T17:59:23Z)
• Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution [62.4972011636884]
  We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework. We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance. We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-02-06T14:12:34Z)
• Separating Content and Style for Unsupervised Image-to-Image Translation [20.44733685446886]
  Unsupervised image-to-image translation aims to learn the mapping between two visual domains with unpaired samples. We propose to separate the content code and style code simultaneously in a unified framework. Based on the correlation between the latent features and the high-level domain-invariant tasks, the proposed framework demonstrates superior performance.
  arXiv  Detail & Related papers  (2021-10-27T12:56:50Z)
• Text as Neural Operator: Image Manipulation by Text Instruction [68.53181621741632]
  In this paper, we study a setting that allows users to edit an image with multiple objects using complex text instructions to add, remove, or change the objects. The inputs of the task are multimodal including (1) a reference image and (2) an instruction in natural language that describes desired modifications to the image. We show that the proposed model performs favorably against recent strong baselines on three public datasets.
  arXiv  Detail & Related papers  (2020-08-11T07:07:10Z)
• COCO-FUNIT: Few-Shot Unsupervised Image Translation with a Content Conditioned Style Encoder [70.23358875904891]
  Unsupervised image-to-image translation aims to learn a mapping of an image in a given domain to an analogous image in a different domain. We propose a new few-shot image translation model, COCO-FUNIT, which computes the style embedding of the example images conditioned on the input image. Our model shows effectiveness in addressing the content loss problem.
  arXiv  Detail & Related papers  (2020-07-15T02:01:14Z)
• Cross-domain Correspondence Learning for Exemplar-based Image Translation [59.35767271091425]
  We present a framework for exemplar-based image translation, which synthesizes a photo-realistic image from the input in a distinct domain. The output has the style (e.g., color, texture) in consistency with the semantically corresponding objects in the exemplar. We show that our method is superior to state-of-the-art methods in terms of image quality significantly.
  arXiv  Detail & Related papers  (2020-04-12T09:10:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.