Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps

• URL: http://arxiv.org/abs/2206.02903v1
• Date: Mon, 6 Jun 2022 21:03:02 GMT
• Title: Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps
• Authors: Seung Wook Kim, Karsten Kreis, Daiqing Li, Antonio Torralba, Sanja Fidler
• Abstract summary: We introduce a generative adversarial network that can simultaneously generate aligned image samples from multiple related domains. We propose Polymorphic-GAN which learns shared features across all domains and a per-domain morph layer to morph shared features according to each domain.
• Score: 94.10535575563092
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern image generative models show remarkable sample quality when trained on a single domain or class of objects. In this work, we introduce a generative adversarial network that can simultaneously generate aligned image samples from multiple related domains. We leverage the fact that a variety of object classes share common attributes, with certain geometric differences. We propose Polymorphic-GAN which learns shared features across all domains and a per-domain morph layer to morph shared features according to each domain. In contrast to previous works, our framework allows simultaneous modelling of images with highly varying geometries, such as images of human faces, painted and artistic faces, as well as multiple different animal faces. We demonstrate that our model produces aligned samples for all domains and show how it can be used for applications such as segmentation transfer and cross-domain image editing, as well as training in low-data regimes. Additionally, we apply our Polymorphic-GAN on image-to-image translation tasks and show that we can greatly surpass previous approaches in cases where the geometric differences between domains are large.

Related papers

• Multi-Scale Multi-Target Domain Adaptation for Angle Closure Classification [50.658613573816254]
  We propose a novel Multi-scale Multi-target Domain Adversarial Network (M2DAN) for angle closure classification. Based on these domain-invariant features at different scales, the deep model trained on the source domain is able to classify angle closure on multiple target domains.
  arXiv  Detail & Related papers  (2022-08-25T15:27:55Z)
• A Domain Gap Aware Generative Adversarial Network for Multi-domain Image Translation [22.47113158859034]
  The paper proposes a unified model to translate images across multiple domains with significant domain gaps. With a single unified generator, the model can maintain consistency over the global shapes as well as the local texture information across multiple domains.
  arXiv  Detail & Related papers  (2021-10-21T00:33:06Z)
• StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators [63.85888518950824]
  We present a text-driven method that allows shifting a generative model to new domains. We show that through natural language prompts and a few minutes of training, our method can adapt a generator across a multitude of domains.
  arXiv  Detail & Related papers  (2021-08-02T14:46:46Z)
• Generating Furry Cars: Disentangling Object Shape & Appearance across Multiple Domains [46.55517346455773]
  We consider the novel task of learning disentangled representations of object shape and appearance across multiple domains. We learn a generative model that learns an intermediate distribution, which borrows a subset of properties from each domain. This challenge requires an accurate disentanglement of object shape, appearance, and background from each domain.
  arXiv  Detail & Related papers  (2021-04-05T17:59:15Z)
• SMILE: Semantically-guided Multi-attribute Image and Layout Editing [154.69452301122175]
  Attribute image manipulation has been a very active topic since the introduction of Generative Adversarial Networks (GANs) We present a multimodal representation that handles all attributes, be it guided by random noise or images, while only using the underlying domain information of the target domain. Our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space.
  arXiv  Detail & Related papers  (2020-10-05T20:15:21Z)
• Latent Normalizing Flows for Many-to-Many Cross-Domain Mappings [76.85673049332428]
  Learned joint representations of images and text form the backbone of several important cross-domain tasks such as image captioning. We propose a novel semi-supervised framework, which models shared information between domains and domain-specific information separately. We demonstrate the effectiveness of our model on diverse tasks, including image captioning and text-to-image synthesis.
  arXiv  Detail & Related papers  (2020-02-16T19:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.