Generative Fields: Uncovering Hierarchical Feature Control for StyleGAN via Inverted Receptive Fields

• URL: http://arxiv.org/abs/2504.17712v1
• Date: Thu, 24 Apr 2025 16:15:02 GMT
• Title: Generative Fields: Uncovering Hierarchical Feature Control for StyleGAN via Inverted Receptive Fields
• Authors: Zhuo He, Paul Henderson, Nicolas Pugeault,
• Abstract summary: This paper introduces the concept of "generative fields" to explain the hierarchical feature synthesis in StyleGAN.<n>We propose a new image editing pipeline for StyleGAN using generative field theory and the channel-wise style latent space S.
• Score: 5.653106385738823
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: StyleGAN has demonstrated the ability of GANs to synthesize highly-realistic faces of imaginary people from random noise. One limitation of GAN-based image generation is the difficulty of controlling the features of the generated image, due to the strong entanglement of the low-dimensional latent space. Previous work that aimed to control StyleGAN with image or text prompts modulated sampling in W latent space, which is more expressive than Z latent space. However, W space still has restricted expressivity since it does not control the feature synthesis directly; also the feature embedding in W space requires a pre-training process to reconstruct the style signal, limiting its application. This paper introduces the concept of "generative fields" to explain the hierarchical feature synthesis in StyleGAN, inspired by the receptive fields of convolution neural networks (CNNs). Additionally, we propose a new image editing pipeline for StyleGAN using generative field theory and the channel-wise style latent space S, utilizing the intrinsic structural feature of CNNs to achieve disentangled control of feature synthesis at synthesis time.

Related papers

• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation [68.42476385214785]
  We propose a novel Spatial-Semantic Map Guided (SSMG) diffusion model that adopts the feature map, derived from the layout, as guidance. SSMG achieves superior generation quality with sufficient spatial and semantic controllability compared to previous works. We also propose the Relation-Sensitive Attention (RSA) and Location-Sensitive Attention (LSA) mechanisms.
  arXiv  Detail & Related papers  (2023-08-20T04:09:12Z)
• Spatial Steerability of GANs via Self-Supervision from Discriminator [123.27117057804732]
  We propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space. Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias. During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects.
  arXiv  Detail & Related papers  (2023-01-20T07:36:29Z)
• Discovering Class-Specific GAN Controls for Semantic Image Synthesis [73.91655061467988]
  We propose a novel method for finding spatially disentangled class-specific directions in the latent space of pretrained SIS models. We show that the latent directions found by our method can effectively control the local appearance of semantic classes.
  arXiv  Detail & Related papers  (2022-12-02T21:39:26Z)
• High-resolution Face Swapping via Latent Semantics Disentanglement [50.23624681222619]
  We present a novel high-resolution hallucination face swapping method using the inherent prior knowledge of a pre-trained GAN model. We explicitly disentangle the latent semantics by utilizing the progressive nature of the generator. We extend our method to video face swapping by enforcing two-temporal constraints on the latent space and the image space.
  arXiv  Detail & Related papers  (2022-03-30T00:33:08Z)
• High-fidelity GAN Inversion with Padding Space [38.9258619444968]
  Inverting a Generative Adversarial Network (GAN) facilitates a wide range of image editing tasks using pre-trained generators. Existing methods typically employ the latent space of GANs as the inversion space yet observe the insufficient recovery of spatial details. We propose to involve the padding space of the generator to complement the latent space with spatial information.
  arXiv  Detail & Related papers  (2022-03-21T16:32:12Z)
• SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing [35.02841064647306]
  StyleGANs provide promising prior models for downstream tasks on image synthesis and editing. We present SemanticStyleGAN, where a generator is trained to model local semantic parts separately and synthesizes images in a compositional way.
  arXiv  Detail & Related papers  (2021-12-04T04:17:11Z)
• Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translation [34.24876359759408]
  We present a novel hierarchical adaptive Diagonal spatial ATtention layers to manipulate the spatial contents from styles in a hierarchical manner. Our method enables coarse-to-fine level disentanglement of spatial contents and styles. Our generator can be easily integrated into the GAN inversion framework so that the content and style of translated images can be flexibly controlled.
  arXiv  Detail & Related papers  (2021-03-30T08:00:13Z)
• Positional Encoding as Spatial Inductive Bias in GANs [97.6622154941448]
  SinGAN shows impressive capability in learning internal patch distribution despite its limited effective receptive field. In this work, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators. We propose a new multi-scale training strategy and demonstrate its effectiveness in the state-of-the-art unconditional generator StyleGAN2.
  arXiv  Detail & Related papers  (2020-12-09T18:27:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.