Neural Crossbreed: Neural Based Image Metamorphosis

• URL: http://arxiv.org/abs/2009.00905v1
• Date: Wed, 2 Sep 2020 08:56:47 GMT
• Title: Neural Crossbreed: Neural Based Image Metamorphosis
• Authors: Sanghun Park, Kwanggyoon Seo, Junyong Noh
• Abstract summary: We propose a feed-forward neural network that can learn a semantic change of input images in a latent space to create the morphing effect. Because the network learns a semantic change, a sequence of meaningful intermediate images can be generated without requiring the user to specify explicit correspondences.
• Score: 11.357156231073862
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We propose Neural Crossbreed, a feed-forward neural network that can learn a semantic change of input images in a latent space to create the morphing effect. Because the network learns a semantic change, a sequence of meaningful intermediate images can be generated without requiring the user to specify explicit correspondences. In addition, the semantic change learning makes it possible to perform the morphing between the images that contain objects with significantly different poses or camera views. Furthermore, just as in conventional morphing techniques, our morphing network can handle shape and appearance transitions separately by disentangling the content and the style transfer for rich usability. We prepare a training dataset for morphing using a pre-trained BigGAN, which generates an intermediate image by interpolating two latent vectors at an intended morphing value. This is the first attempt to address image morphing using a pre-trained generative model in order to learn semantic transformation. The experiments show that Neural Crossbreed produces high quality morphed images, overcoming various limitations associated with conventional approaches. In addition, Neural Crossbreed can be further extended for diverse applications such as multi-image morphing, appearance transfer, and video frame interpolation.

Related papers

• Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks [53.67497327319569]
  We introduce a novel neural rendering technique to solve image-to-3D from a single view. Our approach employs the signed distance function as the surface representation and incorporates generalizable priors through geometry-encoding volumes and HyperNetworks. Our experiments show the advantages of our proposed approach with consistent results and rapid generation.
  arXiv  Detail & Related papers  (2023-12-24T08:42:37Z)
• Cross-Image Attention for Zero-Shot Appearance Transfer [68.43651329067393]
  We introduce a cross-image attention mechanism that implicitly establishes semantic correspondences across images. We harness three mechanisms that either manipulate the noisy latent codes or the model's internal representations throughout the denoising process. Experiments show that our method is effective across a wide range of object categories and is robust to variations in shape, size, and viewpoint.
  arXiv  Detail & Related papers  (2023-11-06T18:33:24Z)
• Neural Implicit Morphing of Face Images [0.7643309077806446]
  Face morphing is a problem in computer graphics with numerous artistic and forensic applications. This task consists of a warping for feature alignment and a blending for a seamless transition between the warped images. We propose to leverage coord-based neural networks to represent such warpings and blendings of face images.
  arXiv  Detail & Related papers  (2023-08-26T14:12:19Z)
• Multimodal Neurons in Pretrained Text-Only Transformers [52.20828443544296]
  We identify "multimodal neurons" that convert visual representations into corresponding text. We show that multimodal neurons operate on specific visual concepts across inputs, and have a systematic causal effect on image captioning.
  arXiv  Detail & Related papers  (2023-08-03T05:27:12Z)
• MorphGANFormer: Transformer-based Face Morphing and De-Morphing [55.211984079735196]
  StyleGAN-based approaches to face morphing are among the leading techniques. We propose a transformer-based alternative to face morphing and demonstrate its superiority to StyleGAN-based methods.
  arXiv  Detail & Related papers  (2023-02-18T19:09:11Z)
• Multiview Regenerative Morphing with Dual Flows [20.062713286961326]
  We propose a novel approach called Multiview Regenerative Morphing. It formulates the morphing process as an optimization to solve for rigid and optimal-transport. Our approach does not rely on user-specified correspondences or 2D/3D input meshes.
  arXiv  Detail & Related papers  (2022-08-02T07:22:48Z)
• Image Morphing with Perceptual Constraints and STN Alignment [70.38273150435928]
  We propose a conditional GAN morphing framework operating on a pair of input images. A special training protocol produces sequences of frames, combined with a perceptual similarity loss, promote smooth transformation over time. We provide comparisons to classic as well as latent space morphing techniques, and demonstrate that, given a set of images for self-supervision, our network learns to generate visually pleasing morphing effects.
  arXiv  Detail & Related papers  (2020-04-29T10:49:10Z)
• Fine-grained Image-to-Image Transformation towards Visual Recognition [102.51124181873101]
  We aim at transforming an image with a fine-grained category to synthesize new images that preserve the identity of the input image. We adopt a model based on generative adversarial networks to disentangle the identity related and unrelated factors of an image. Experiments on the CompCars and Multi-PIE datasets demonstrate that our model preserves the identity of the generated images much better than the state-of-the-art image-to-image transformation models.
  arXiv  Detail & Related papers  (2020-01-12T05:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.