Editing in Style: Uncovering the Local Semantics of GANs

• URL: http://arxiv.org/abs/2004.14367v2
• Date: Thu, 21 May 2020 13:01:07 GMT
• Title: Editing in Style: Uncovering the Local Semantics of GANs
• Authors: Edo Collins, Raja Bala, Bob Price, Sabine S\"usstrunk
• Abstract summary: We introduce a simple and effective method for making local, semantically-aware edits to a target output image. This is accomplished by borrowing elements from a source image, also a GAN output, via a novel manipulation of style vectors. We measure the locality and photorealism of the edits produced by our method, and find that it accomplishes both.
• Score: 6.342949222955067
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While the quality of GAN image synthesis has improved tremendously in recent years, our ability to control and condition the output is still limited. Focusing on StyleGAN, we introduce a simple and effective method for making local, semantically-aware edits to a target output image. This is accomplished by borrowing elements from a source image, also a GAN output, via a novel manipulation of style vectors. Our method requires neither supervision from an external model, nor involves complex spatial morphing operations. Instead, it relies on the emergent disentanglement of semantic objects that is learned by StyleGAN during its training. Semantic editing is demonstrated on GANs producing human faces, indoor scenes, cats, and cars. We measure the locality and photorealism of the edits produced by our method, and find that it accomplishes both.

Related papers

• DiffUHaul: A Training-Free Method for Object Dragging in Images [78.93531472479202]
  We propose a training-free method, dubbed DiffUHaul, for the object dragging task. We first apply attention masking in each denoising step to make the generation more disentangled across different objects. In the early denoising steps, we interpolate the attention features between source and target images to smoothly fuse new layouts with the original appearance.
  arXiv  Detail & Related papers  (2024-06-03T17:59:53Z)
• ZONE: Zero-Shot Instruction-Guided Local Editing [56.56213730578504]
  We propose a Zero-shot instructiON-guided local image Editing approach, termed ZONE. We first convert the editing intent from the user-provided instruction into specific image editing regions through InstructPix2Pix. We then propose a Region-IoU scheme for precise image layer extraction from an off-the-shelf segment model.
  arXiv  Detail & Related papers  (2023-12-28T02:54:34Z)
• Spatial Steerability of GANs via Self-Supervision from Discriminator [123.27117057804732]
  We propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space. Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias. During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects.
  arXiv  Detail & Related papers  (2023-01-20T07:36:29Z)
• Video2StyleGAN: Disentangling Local and Global Variations in a Video [68.70889857355678]
  StyleGAN has emerged as a powerful paradigm for facial editing, providing disentangled controls over age, expression, illumination, etc. We introduce Video2StyleGAN that takes a target image and driving video(s) to reenact the local and global locations and expressions from the driving video in the identity of the target image.
  arXiv  Detail & Related papers  (2022-05-27T14:18:19Z)
• FEAT: Face Editing with Attention [70.89233432407305]
  We build on the StyleGAN generator and present a method that explicitly encourages face manipulation to focus on the intended regions. During the generation of the edited image, the attention map serves as a mask that guides a blending between the original features and the modified ones.
  arXiv  Detail & Related papers  (2022-02-06T06:07:34Z)
• Perceptually Validated Precise Local Editing for Facial Action Units with StyleGAN [3.8149289266694466]
  We build a solution based on StyleGAN, which has been used extensively for semantic manipulation of faces. We show that a naive strategy to perform editing in the latent space results in undesired coupling between certain action units. We validate the effectiveness of our local editing method through perception experiments conducted with 23 subjects.
  arXiv  Detail & Related papers  (2021-07-26T12:21:37Z)
• Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks [15.253043666814413]
  We propose an effective approach, termed as LoGAN, to support local editing of the output image. We are able to seamlessly remove, insert, shift, and rotate the individual objects inside a room. Our method can completely clear out a room and then refurnish it with customized furniture and styles.
  arXiv  Detail & Related papers  (2021-05-18T01:31:49Z)
• PIE: Portrait Image Embedding for Semantic Control [82.69061225574774]
  We present the first approach for embedding real portrait images in the latent space of StyleGAN. We use StyleRig, a pretrained neural network that maps the control space of a 3D morphable face model to the latent space of the GAN. An identity energy preservation term allows spatially coherent edits while maintaining facial integrity.
  arXiv  Detail & Related papers  (2020-09-20T17:53:51Z)
• StyleGAN2 Distillation for Feed-forward Image Manipulation [5.5080625617632]
  StyleGAN2 is a state-of-the-art network in generating realistic images. We propose a way to distill a particular image manipulation of StyleGAN2 into image-to-image network trained in paired way.
  arXiv  Detail & Related papers  (2020-03-07T14:02:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.