Dual-Domain Image Synthesis using Segmentation-Guided GAN

• URL: http://arxiv.org/abs/2204.09015v1
• Date: Tue, 19 Apr 2022 17:25:54 GMT
• Title: Dual-Domain Image Synthesis using Segmentation-Guided GAN
• Authors: Dena Bazazian, Andrew Calway, Dima Damen
• Abstract summary: We introduce a segmentation-guided approach to synthesise images that integrate features from two distinct domains. Images synthesised by our dual-domain model belong to one domain within the semantic mask, and to another in the rest of the image.
• Score: 33.00724627120716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a segmentation-guided approach to synthesise images that integrate features from two distinct domains. Images synthesised by our dual-domain model belong to one domain within the semantic mask, and to another in the rest of the image - smoothly integrated. We build on the successes of few-shot StyleGAN and single-shot semantic segmentation to minimise the amount of training required in utilising two domains. The method combines a few-shot cross-domain StyleGAN with a latent optimiser to achieve images containing features of two distinct domains. We use a segmentation-guided perceptual loss, which compares both pixel-level and activations between domain-specific and dual-domain synthetic images. Results demonstrate qualitatively and quantitatively that our model is capable of synthesising dual-domain images on a variety of objects (faces, horses, cats, cars), domains (natural, caricature, sketches) and part-based masks (eyes, nose, mouth, hair, car bonnet). The code is publicly available at: https://github.com/denabazazian/Dual-Domain-Synthesis.

Related papers

• SynCDR : Training Cross Domain Retrieval Models with Synthetic Data [69.26882668598587]
  In cross-domain retrieval, a model is required to identify images from the same semantic category across two visual domains. We show how to generate synthetic data to fill in these missing category examples across domains. Our best SynCDR model can outperform prior art by up to 15%.
  arXiv  Detail & Related papers  (2023-12-31T08:06:53Z)
• Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation [65.78246406460305]
  compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
  arXiv  Detail & Related papers  (2023-08-28T14:43:36Z)
• Domain-Scalable Unpaired Image Translation via Latent Space Anchoring [88.7642967393508]
  Unpaired image-to-image translation (UNIT) aims to map images between two visual domains without paired training data. We propose a new domain-scalable UNIT method, termed as latent space anchoring. Our method anchors images of different domains to the same latent space of frozen GANs by learning lightweight encoder and regressor models. In the inference phase, the learned encoders and decoders of different domains can be arbitrarily combined to translate images between any two domains without fine-tuning.
  arXiv  Detail & Related papers  (2023-06-26T17:50:02Z)
• Disentangled Unsupervised Image Translation via Restricted Information Flow [61.44666983942965]
  Many state-of-art methods hard-code the desired shared-vs-specific split into their architecture. We propose a new method that does not rely on inductive architectural biases. We show that the proposed method achieves consistently high manipulation accuracy across two synthetic and one natural dataset.
  arXiv  Detail & Related papers  (2021-11-26T00:27:54Z)
• Crossing-Domain Generative Adversarial Networks for Unsupervised Multi-Domain Image-to-Image Translation [12.692904507625036]
  We propose a general framework for unsupervised image-to-image translation across multiple domains. Our proposed framework consists of a pair of encoders along with a pair of GANs which learns high-level features across different domains to generate diverse and realistic samples from.
  arXiv  Detail & Related papers  (2020-08-27T01:54:07Z)
• SoloGAN: Multi-domain Multimodal Unpaired Image-to-Image Translation via a Single Generative Adversarial Network [4.7344504314446345]
  We present a flexible and general SoloGAN model for efficient multimodal I2I translation among multiple domains with unpaired data. In contrast to existing methods, the SoloGAN algorithm uses a single projection discriminator with an additional auxiliary classifier and shares the encoder and generator for all domains.
  arXiv  Detail & Related papers  (2020-08-04T16:31:15Z)
• GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling [66.50914391679375]
  Unsupervised image-to-image translation (UNIT) aims at learning a mapping between several visual domains by using unpaired training images. Recent studies have shown remarkable success for multiple domains but they suffer from two main limitations. We propose a method named GMM-UNIT, which is based on a content-attribute disentangled representation where the space is fitted with a GMM.
  arXiv  Detail & Related papers  (2020-03-15T10:18:56Z)
• Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings [76.85673049332428]
  Learned joint representations of images and text form the backbone of several important cross-domain tasks such as image captioning. We propose a novel semi-supervised framework, which models shared information between domains and domain-specific information separately. We demonstrate the effectiveness of our model on diverse tasks, including image captioning and text-to-image synthesis.
  arXiv  Detail & Related papers  (2020-02-16T19:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.