State-of-the-Art in the Architecture, Methods and Applications of StyleGAN

• URL: http://arxiv.org/abs/2202.14020v1
• Date: Mon, 28 Feb 2022 18:42:04 GMT
• Title: State-of-the-Art in the Architecture, Methods and Applications of StyleGAN
• Authors: Amit H. Bermano and Rinon Gal and Yuval Alaluf and Ron Mokady and Yotam Nitzan and Omer Tov and Or Patashnik and Daniel Cohen-Or
• Abstract summary: State-of-the-art report covers the StyleGAN architecture, and the ways it has been employed since its conception. StyleGAN's learned latent space is surprisingly well-behaved and remarkably disentangled. The control offered by StyleGAN is inherently limited to the generator's learned distribution.
• Score: 41.359306557243706
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Generative Adversarial Networks (GANs) have established themselves as a prevalent approach to image synthesis. Of these, StyleGAN offers a fascinating case study, owing to its remarkable visual quality and an ability to support a large array of downstream tasks. This state-of-the-art report covers the StyleGAN architecture, and the ways it has been employed since its conception, while also analyzing its severe limitations. It aims to be of use for both newcomers, who wish to get a grasp of the field, and for more experienced readers that might benefit from seeing current research trends and existing tools laid out. Among StyleGAN's most interesting aspects is its learned latent space. Despite being learned with no supervision, it is surprisingly well-behaved and remarkably disentangled. Combined with StyleGAN's visual quality, these properties gave rise to unparalleled editing capabilities. However, the control offered by StyleGAN is inherently limited to the generator's learned distribution, and can only be applied to images generated by StyleGAN itself. Seeking to bring StyleGAN's latent control to real-world scenarios, the study of GAN inversion and latent space embedding has quickly gained in popularity. Meanwhile, this same study has helped shed light on the inner workings and limitations of StyleGAN. We map out StyleGAN's impressive story through these investigations, and discuss the details that have made StyleGAN the go-to generator. We further elaborate on the visual priors StyleGAN constructs, and discuss their use in downstream discriminative tasks. Looking forward, we point out StyleGAN's limitations and speculate on current trends and promising directions for future research, such as task and target specific fine-tuning.

Related papers

• HiCAST: Highly Customized Arbitrary Style Transfer with Adapter Enhanced Diffusion Models [84.12784265734238]
  The goal of Arbitrary Style Transfer (AST) is injecting the artistic features of a style reference into a given image/video. We propose HiCAST, which is capable of explicitly customizing the stylization results according to various source of semantic clues. A novel learning objective is leveraged for video diffusion model training, which significantly improve cross-frame temporal consistency.
  arXiv  Detail & Related papers  (2024-01-11T12:26:23Z)
• Spatial Steerability of GANs via Self-Supervision from Discriminator [123.27117057804732]
  We propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space. Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias. During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects.
  arXiv  Detail & Related papers  (2023-01-20T07:36:29Z)
• Face Generation and Editing with StyleGAN: A Survey [0.12362527696478205]
  The survey covers the evolution of StyleGAN, from PGGAN to StyleGAN3, and explores relevant topics such as suitable metrics for training, different latent representations, GAN inversion to latent spaces of StyleGAN, face image editing, cross-domain face stylization, face restoration, and even Deepfake applications.
  arXiv  Detail & Related papers  (2022-12-18T15:04:31Z)
• Mind the Gap in Distilling StyleGANs [100.58444291751015]
  StyleGAN family is one of the most popular Generative Adversarial Networks (GANs) for unconditional generation. This paper provides a comprehensive study of distilling from the popular StyleGAN-like architecture.
  arXiv  Detail & Related papers  (2022-08-18T14:18:29Z)
• Style-Based Global Appearance Flow for Virtual Try-On [119.95115739956661]
  A novel global appearance flow estimation model is proposed in this work. Experiment results on a popular virtual try-on benchmark show that our method achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-04-03T10:58:04Z)
• Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs [0.0]
  StyleGAN2 has enabled various image generation and manipulation tasks due to its rich and disentangled latent spaces. We design a novel submodular framework that finds the most representative and diverse subset of directions in the latent space of StyleGAN2. Our framework promotes diversity by using the notion of clusters and can be efficiently solved with a greedy optimization scheme.
  arXiv  Detail & Related papers  (2022-03-16T10:35:41Z)
• DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN [30.465955123686335]
  We introduce SC-StyleGAN, which injects spatial constraints to the original StyleGAN generation process. Based on SC-StyleGAN, we present DrawingInStyles, a novel drawing interface for non-professional users to easily produce high-quality, photo-realistic face images.
  arXiv  Detail & Related papers  (2022-03-05T14:54:07Z)
• Third Time's the Charm? Image and Video Editing with StyleGAN3 [70.36056009463738]
  StyleGAN is arguably one of the most intriguing and well-studied generative models. We explore the recent StyleGAN3 architecture, compare it to its predecessor, and investigate its unique advantages, as well as drawbacks.
  arXiv  Detail & Related papers  (2022-01-31T18:44:59Z)
• Fine-Grained Control of Artistic Styles in Image Generation [24.524863555822837]
  generative models and adversarial training have enabled artificially generating artworks in various artistic styles. We propose to capture the continuous spectrum of styles and apply it to a style generation task. Our method can be used with common generative adversarial networks (such as StyleGAN)
  arXiv  Detail & Related papers  (2021-10-19T21:51:52Z)
• Editing in Style: Uncovering the Local Semantics of GANs [6.342949222955067]
  We introduce a simple and effective method for making local, semantically-aware edits to a target output image. This is accomplished by borrowing elements from a source image, also a GAN output, via a novel manipulation of style vectors. We measure the locality and photorealism of the edits produced by our method, and find that it accomplishes both.
  arXiv  Detail & Related papers  (2020-04-29T17:45:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.