Rethinking Image Skip Connections in StyleGAN2

• URL: http://arxiv.org/abs/2407.05527v1
• Date: Mon, 8 Jul 2024 00:21:17 GMT
• Title: Rethinking Image Skip Connections in StyleGAN2
• Authors: Seung Park, Yong-Goo Shin,
• Abstract summary: StyleGAN models have gained significant traction in the field of image synthesis. The adoption of image skip connection is favored over the traditional residual connection. We introduce the image squeeze connection, which significantly improves the quality of image synthesis.
• Score: 5.929956715430167
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various models based on StyleGAN have gained significant traction in the field of image synthesis, attributed to their robust training stability and superior performances. Within the StyleGAN framework, the adoption of image skip connection is favored over the traditional residual connection. However, this preference is just based on empirical observations; there has not been any in-depth mathematical analysis on it yet. To rectify this situation, this brief aims to elucidate the mathematical meaning of the image skip connection and introduce a groundbreaking methodology, termed the image squeeze connection, which significantly improves the quality of image synthesis. Specifically, we analyze the image skip connection technique to reveal its problem and introduce the proposed method which not only effectively boosts the GAN performance but also reduces the required number of network parameters. Extensive experiments on various datasets demonstrate that the proposed method consistently enhances the performance of state-of-the-art models based on StyleGAN. We believe that our findings represent a vital advancement in the field of image synthesis, suggesting a novel direction for future research and applications.

Related papers

• Enhancing Conditional Image Generation with Explainable Latent Space Manipulation [0.0]
  This paper proposes a novel approach to achieve fidelity to a reference image while adhering to conditional prompts. We analyze the cross attention maps of the cross attention layers and gradients for the denoised latent vector. Using this information, we create masks at specific timesteps during denoising to preserve subjects while seamlessly integrating the reference image features.
  arXiv  Detail & Related papers  (2024-08-29T03:12:04Z)
• Information Theoretic Text-to-Image Alignment [49.396917351264655]
  We present a novel method that relies on an information-theoretic alignment measure to steer image generation. Our method is on-par or superior to the state-of-the-art, yet requires nothing but a pre-trained denoising network to estimate MI.
  arXiv  Detail & Related papers  (2024-05-31T12:20:02Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis [65.7968515029306]
  We propose a novel Coarse-to-Fine Latent Diffusion (CFLD) method for Pose-Guided Person Image Synthesis (PGPIS) A perception-refined decoder is designed to progressively refine a set of learnable queries and extract semantic understanding of person images as a coarse-grained prompt.
  arXiv  Detail & Related papers  (2024-02-28T06:07:07Z)
• IRGen: Generative Modeling for Image Retrieval [82.62022344988993]
  In this paper, we present a novel methodology, reframing image retrieval as a variant of generative modeling. We develop our model, dubbed IRGen, to address the technical challenge of converting an image into a concise sequence of semantic units. Our model achieves state-of-the-art performance on three widely-used image retrieval benchmarks and two million-scale datasets.
  arXiv  Detail & Related papers  (2023-03-17T17:07:36Z)
• A Novel Sampling Scheme for Text- and Image-Conditional Image Synthesis in Quantized Latent Spaces [0.7340845393655052]
  We propose a streamlined approach for text-to-image generation, which encompasses both the training paradigm and the sampling process. Despite its remarkable simplicity, our method yields aesthetically pleasing images with few sampling iterations. To demonstrate the efficacy of this approach in achieving outcomes comparable to existing works, we have trained a one-billion parameter text-conditional model.
  arXiv  Detail & Related papers  (2022-11-14T11:52:55Z)
• A Visual Navigation Perspective for Category-Level Object Pose Estimation [41.60364392204057]
  This paper studies category-level object pose estimation based on a single monocular image. Recent advances in pose-aware generative models have paved the way for addressing this challenging task using analysis-by-synthesis.
  arXiv  Detail & Related papers  (2022-03-25T10:57:37Z)
• Enhancing Photorealism Enhancement [83.88433283714461]
  We present an approach to enhancing the realism of synthetic images using a convolutional network. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We report substantial gains in stability and realism in comparison to recent image-to-image translation methods.
  arXiv  Detail & Related papers  (2021-05-10T19:00:49Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.