FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations
- URL: http://arxiv.org/abs/2202.06240v1
- Date: Sun, 13 Feb 2022 07:39:48 GMT
- Title: FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations
- Authors: Cemre Karakas, Alara Dirik, Eylul Yalcinkaya, Pinar Yanardag
- Abstract summary: We propose a method for modifying a pre-trained StyleGAN2 model that can be used to generate a balanced set of images.
Our method successfully debiases the GAN model within a few minutes without compromising the quality of the generated images.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Recent advances in generative adversarial networks have shown that it is
possible to generate high-resolution and hyperrealistic images. However, the
images produced by GANs are only as fair and representative as the datasets on
which they are trained. In this paper, we propose a method for directly
modifying a pre-trained StyleGAN2 model that can be used to generate a balanced
set of images with respect to one (e.g., eyeglasses) or more attributes (e.g.,
gender and eyeglasses). Our method takes advantage of the style space of the
StyleGAN2 model to perform disentangled control of the target attributes to be
debiased. Our method does not require training additional models and directly
debiases the GAN model, paving the way for its use in various downstream
applications. Our experiments show that our method successfully debiases the
GAN model within a few minutes without compromising the quality of the
generated images. To promote fair generative models, we share the code and
debiased models at http://catlab-team.github.io/fairstyle.
Related papers
- LatentSwap: An Efficient Latent Code Mapping Framework for Face Swapping [41.59473130977597]
We propose LatentSwap, a framework generating a face swap latent code of a given generator.
Our framework is light and does not require datasets besides employing the pre-trained models.
We show that our framework is applicable to other generators such as StyleNeRF, paving a way to 3D-aware face swapping.
arXiv Detail & Related papers (2024-02-28T14:17:32Z) - Direct Consistency Optimization for Compositional Text-to-Image
Personalization [73.94505688626651]
Text-to-image (T2I) diffusion models, when fine-tuned on a few personal images, are able to generate visuals with a high degree of consistency.
We propose to fine-tune the T2I model by maximizing consistency to reference images, while penalizing the deviation from the pretrained model.
arXiv Detail & Related papers (2024-02-19T09:52:41Z) - Customize StyleGAN with One Hand Sketch [0.0]
We propose a framework to control StyleGAN imagery with a single user sketch.
We learn a conditional distribution in the latent space of a pre-trained StyleGAN model via energy-based learning.
Our model can generate multi-modal images semantically aligned with the input sketch.
arXiv Detail & Related papers (2023-10-29T09:32:33Z) - MODIFY: Model-driven Face Stylization without Style Images [77.24793103549158]
Existing face stylization methods always acquire the presence of the target (style) domain during the translation process.
We propose a new method called MODel-drIven Face stYlization (MODIFY), which relies on the generative model to bypass the dependence of the target images.
Experimental results on several different datasets validate the effectiveness of MODIFY for unsupervised face stylization.
arXiv Detail & Related papers (2023-03-17T08:35:17Z) - Spatial Steerability of GANs via Self-Supervision from Discriminator [123.27117057804732]
We propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space.
Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias.
During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects.
arXiv Detail & Related papers (2023-01-20T07:36:29Z) - Uncovering the Disentanglement Capability in Text-to-Image Diffusion
Models [60.63556257324894]
A key desired property of image generative models is the ability to disentangle different attributes.
We propose a simple, light-weight image editing algorithm where the mixing weights of the two text embeddings are optimized for style matching and content preservation.
Experiments show that the proposed method can modify a wide range of attributes, with the performance outperforming diffusion-model-based image-editing algorithms.
arXiv Detail & Related papers (2022-12-16T19:58:52Z) - Lafite2: Few-shot Text-to-Image Generation [132.14211027057766]
We propose a novel method for pre-training text-to-image generation model on image-only datasets.
It considers a retrieval-then-optimization procedure to synthesize pseudo text features.
It can be beneficial to a wide range of settings, including the few-shot, semi-supervised and fully-supervised learning.
arXiv Detail & Related papers (2022-10-25T16:22:23Z) - Implementing and Experimenting with Diffusion Models for Text-to-Image
Generation [0.0]
Two models, DALL-E 2 and Imagen, have demonstrated that highly photorealistic images could be generated from a simple textual description of an image.
Text-to-image models require exceptionally large amounts of computational resources to train, as well as handling huge datasets collected from the internet.
This thesis contributes by reviewing the different approaches and techniques used by these models, and then by proposing our own implementation of a text-to-image model.
arXiv Detail & Related papers (2022-09-22T12:03:33Z) - InvGAN: Invertible GANs [88.58338626299837]
InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model.
This allows us to perform image inpainting, merging, and online data augmentation.
arXiv Detail & Related papers (2021-12-08T21:39:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.