Generating Furry Cars: Disentangling Object Shape & Appearance across Multiple Domains

• URL: http://arxiv.org/abs/2104.02052v1
• Date: Mon, 5 Apr 2021 17:59:15 GMT
• Title: Generating Furry Cars: Disentangling Object Shape & Appearance across Multiple Domains
• Authors: Utkarsh Ojha, Krishna Kumar Singh, Yong Jae Lee
• Abstract summary: We consider the novel task of learning disentangled representations of object shape and appearance across multiple domains. We learn a generative model that learns an intermediate distribution, which borrows a subset of properties from each domain. This challenge requires an accurate disentanglement of object shape, appearance, and background from each domain.
• Score: 46.55517346455773
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We consider the novel task of learning disentangled representations of object shape and appearance across multiple domains (e.g., dogs and cars). The goal is to learn a generative model that learns an intermediate distribution, which borrows a subset of properties from each domain, enabling the generation of images that did not exist in any domain exclusively. This challenging problem requires an accurate disentanglement of object shape, appearance, and background from each domain, so that the appearance and shape factors from the two domains can be interchanged. We augment an existing approach that can disentangle factors within a single domain but struggles to do so across domains. Our key technical contribution is to represent object appearance with a differentiable histogram of visual features, and to optimize the generator so that two images with the same latent appearance factor but different latent shape factors produce similar histograms. On multiple multi-domain datasets, we demonstrate our method leads to accurate and consistent appearance and shape transfer across domains.

Related papers

• Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation [65.78246406460305]
  compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
  arXiv  Detail & Related papers  (2023-08-28T14:43:36Z)
• Polymorphic-GAN: Generating Aligned Samples across Multiple Domains with Learned Morph Maps [94.10535575563092]
  We introduce a generative adversarial network that can simultaneously generate aligned image samples from multiple related domains. We propose Polymorphic-GAN which learns shared features across all domains and a per-domain morph layer to morph shared features according to each domain.
  arXiv  Detail & Related papers  (2022-06-06T21:03:02Z)
• Domain Invariant Masked Autoencoders for Self-supervised Learning from Multi-domains [73.54897096088149]
  We propose a Domain-invariant Masked AutoEncoder (DiMAE) for self-supervised learning from multi-domains. The core idea is to augment the input image with style noise from different domains and then reconstruct the image from the embedding of the augmented image. Experiments on PACS and DomainNet illustrate that DiMAE achieves considerable gains compared with recent state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-10T09:49:40Z)
• A Domain Gap Aware Generative Adversarial Network for Multi-domain Image Translation [22.47113158859034]
  The paper proposes a unified model to translate images across multiple domains with significant domain gaps. With a single unified generator, the model can maintain consistency over the global shapes as well as the local texture information across multiple domains.
  arXiv  Detail & Related papers  (2021-10-21T00:33:06Z)
• DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation [23.588766224169493]
  DRANet is a network architecture that disentangles image representations and transfers the visual attributes in a latent space for unsupervised cross-domain adaptation. Our model encodes individual representations of content (scene structure) and style (artistic appearance) from both source and target images. It adapts the domain by incorporating the transferred style factor into the content factor along with learnable weights specified for each domain.
  arXiv  Detail & Related papers  (2021-03-24T18:54:23Z)
• Domain Adaptation for Learning Generator from Paired Few-Shot Data [72.04430033118426]
  We propose a Paired Few-shot GAN (PFS-GAN) model for learning generators with sufficient source data and a few target data. Our method has better quantitative and qualitative results on the generated target-domain data with higher diversity in comparison to several baselines.
  arXiv  Detail & Related papers  (2021-02-25T10:11:44Z)
• Variational Interaction Information Maximization for Cross-domain Disentanglement [34.08140408283391]
  Cross-domain disentanglement is the problem of learning representations partitioned into domain-invariant and domain-specific representations. We cast the simultaneous learning of domain-invariant and domain-specific representations as a joint objective of multiple information constraints. We show that our model achieves the state-of-the-art performance in the zero-shot sketch based image retrieval task.
  arXiv  Detail & Related papers  (2020-12-08T07:11:35Z)
• Shape-aware Generative Adversarial Networks for Attribute Transfer [10.786099674296986]
  We introduce a shape-aware GAN model that is able to preserve shape when transferring attributes. Compared to other state-of-art GANs-based image-to-image translation models, the model we propose is able to generate more visually appealing results.
  arXiv  Detail & Related papers  (2020-10-11T14:52:32Z)
• SMILE: Semantically-guided Multi-attribute Image and Layout Editing [154.69452301122175]
  Attribute image manipulation has been a very active topic since the introduction of Generative Adversarial Networks (GANs) We present a multimodal representation that handles all attributes, be it guided by random noise or images, while only using the underlying domain information of the target domain. Our method is capable of adding, removing or changing either fine-grained or coarse attributes by using an image as a reference or by exploring the style distribution space.
  arXiv  Detail & Related papers  (2020-10-05T20:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.