Attribute-specific Control Units in StyleGAN for Fine-grained Image Manipulation

• URL: http://arxiv.org/abs/2111.13010v1
• Date: Thu, 25 Nov 2021 10:42:10 GMT
• Title: Attribute-specific Control Units in StyleGAN for Fine-grained Image Manipulation
• Authors: Rui Wang, Jian Chen, Gang Yu, Li Sun, Changqian Yu, Changxin Gao, Nong Sang
• Abstract summary: We discover attribute-specific control units, which consist of multiple channels of feature maps and modulation styles. Specifically, we collaboratively manipulate the modulation style channels and feature maps in control units to obtain the semantic and spatial disentangled controls. We move the modulation style along a specific sparse direction vector and replace the filter-wise styles used to compute the feature maps to manipulate these control units.
• Score: 57.99007520795998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image manipulation with StyleGAN has been an increasing concern in recent years.Recent works have achieved tremendous success in analyzing several semantic latent spaces to edit the attributes of the generated images.However, due to the limited semantic and spatial manipulation precision in these latent spaces, the existing endeavors are defeated in fine-grained StyleGAN image manipulation, i.e., local attribute translation.To address this issue, we discover attribute-specific control units, which consist of multiple channels of feature maps and modulation styles. Specifically, we collaboratively manipulate the modulation style channels and feature maps in control units rather than individual ones to obtain the semantic and spatial disentangled controls. Furthermore, we propose a simple yet effective method to detect the attribute-specific control units. We move the modulation style along a specific sparse direction vector and replace the filter-wise styles used to compute the feature maps to manipulate these control units. We evaluate our proposed method in various face attribute manipulation tasks. Extensive qualitative and quantitative results demonstrate that our proposed method performs favorably against the state-of-the-art methods. The manipulation results of real images further show the effectiveness of our method.

Related papers

• Localizing Object-level Shape Variations with Text-to-Image Diffusion Models [60.422435066544814]
  We present a technique to generate a collection of images that depicts variations in the shape of a specific object. A particular challenge when generating object variations is accurately localizing the manipulation applied over the object's shape. To localize the image-space operation, we present two techniques that use the self-attention layers in conjunction with the cross-attention layers.
  arXiv  Detail & Related papers  (2023-03-20T17:45:08Z)
• Exploring the Effectiveness of Mask-Guided Feature Modulation as a Mechanism for Localized Style Editing of Real Images [33.018300966769516]
  We present the SemanticStyle Autoencoder (SSAE), a deep Generative Autoencoder model that leverages semantic mask-guided latent space manipulation. This work shall serve as a guiding primer for future work.
  arXiv  Detail & Related papers  (2022-11-21T07:36:20Z)
• Everything is There in Latent Space: Attribute Editing and Attribute Style Manipulation by StyleGAN Latent Space Exploration [39.18239951479647]
  We present Few-shot Latent-based Attribute Manipulation and Editing (FLAME) FLAME is a framework to perform highly controlled image editing by latent space manipulation. We generate diverse attribute styles in disentangled manner.
  arXiv  Detail & Related papers  (2022-07-20T12:40:32Z)
• DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing [12.80013698957431]
  A Dynamic Style Manipulation Network (DyStyle) is proposed to perform attribute-conditioned style editing. A novel easy-to-hard training procedure is introduced for efficient and stable training of the DyStyle network. Our approach demonstrates fine-grained disentangled edits along multiple numeric and binary attributes.
  arXiv  Detail & Related papers  (2021-09-22T13:50:51Z)
• Surrogate Gradient Field for Latent Space Manipulation [4.880243880711163]
  Generative adversarial networks (GANs) can generate high-quality images from sampled latent codes. Recent works attempt to edit an image by manipulating its underlying latent code, but rarely go beyond the basic task of attribute adjustment. We propose the first method that enables manipulation with multidimensional condition such as keypoints and captions.
  arXiv  Detail & Related papers  (2021-04-19T06:15:06Z)
• StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery [71.1862388442953]
  We develop a text-based interface for StyleGAN image manipulation. We first introduce an optimization scheme that utilizes a CLIP-based loss to modify an input latent vector in response to a user-provided text prompt. Next, we describe a latent mapper that infers a text-guided latent manipulation step for a given input image, allowing faster and more stable text-based manipulation.
  arXiv  Detail & Related papers  (2021-03-31T17:51:25Z)
• Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation [136.53288628437355]
  Controllable semantic image editing enables a user to change entire image attributes with few clicks. Current approaches often suffer from attribute edits that are entangled, global image identity changes, and diminished photo-realism. We propose quantitative evaluation strategies for measuring controllable editing performance, unlike prior work which primarily focuses on qualitative evaluation.
  arXiv  Detail & Related papers  (2021-02-01T21:38:36Z)
• StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation [45.20783737095007]
  We explore and analyze the latent style space of StyleGAN2, a state-of-the-art architecture for image generation. StyleSpace is significantly more disentangled than the other intermediate latent spaces explored by previous works. Our findings pave the way to semantically meaningful and well-disentangled image manipulations via simple and intuitive interfaces.
  arXiv  Detail & Related papers  (2020-11-25T15:00:33Z)
• Style Intervention: How to Achieve Spatial Disentanglement with Style-based Generators? [100.60938767993088]
  We propose a lightweight optimization-based algorithm which could adapt to arbitrary input images and render natural translation effects under flexible objectives. We verify the performance of the proposed framework in facial attribute editing on high-resolution images, where both photo-realism and consistency are required.
  arXiv  Detail & Related papers  (2020-11-19T07:37:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.