PS-StyleGAN: Illustrative Portrait Sketching using Attention-Based Style Adaptation

• URL: http://arxiv.org/abs/2409.00345v1
• Date: Sat, 31 Aug 2024 04:22:45 GMT
• Title: PS-StyleGAN: Illustrative Portrait Sketching using Attention-Based Style Adaptation
• Authors: Kushal Kumar Jain, Ankith Varun J, Anoop Namboodiri,
• Abstract summary: Portrait sketching involves capturing identity specific attributes of a real face with abstract lines and shades. This paper introduces textbfPortrait Sketching StyleGAN (PS-StyleGAN), a style transfer approach tailored for portrait sketch synthesis. We leverage the semantic $W+$ latent space of StyleGAN to generate portrait sketches, allowing us to make meaningful edits, like pose and expression alterations, without compromising identity.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Portrait sketching involves capturing identity specific attributes of a real face with abstract lines and shades. Unlike photo-realistic images, a good portrait sketch generation method needs selective attention to detail, making the problem challenging. This paper introduces \textbf{Portrait Sketching StyleGAN (PS-StyleGAN)}, a style transfer approach tailored for portrait sketch synthesis. We leverage the semantic $W+$ latent space of StyleGAN to generate portrait sketches, allowing us to make meaningful edits, like pose and expression alterations, without compromising identity. To achieve this, we propose the use of Attentive Affine transform blocks in our architecture, and a training strategy that allows us to change StyleGAN's output without finetuning it. These blocks learn to modify style latent code by paying attention to both content and style latent features, allowing us to adapt the outputs of StyleGAN in an inversion-consistent manner. Our approach uses only a few paired examples ($\sim 100$) to model a style and has a short training time. We demonstrate PS-StyleGAN's superiority over the current state-of-the-art methods on various datasets, qualitatively and quantitatively.

Related papers

• Portrait Diffusion: Training-free Face Stylization with Chain-of-Painting [64.43760427752532]
  Face stylization refers to the transformation of a face into a specific portrait style. Current methods require the use of example-based adaptation approaches to fine-tune pre-trained generative models. This paper proposes a training-free face stylization framework, named Portrait Diffusion.
  arXiv  Detail & Related papers  (2023-12-03T06:48:35Z)
• Face Cartoonisation For Various Poses Using StyleGAN [0.7673339435080445]
  This paper presents an innovative approach to achieve face cartoonisation while preserving the original identity and accommodating various poses. We achieve this by introducing an encoder that captures both pose and identity information from images and generates a corresponding embedding within the StyleGAN latent space. We show by extensive experimentation how our encoder adapts the StyleGAN output to better preserve identity when the objective is cartoonisation.
  arXiv  Detail & Related papers  (2023-09-26T13:10:25Z)
• StylerDALLE: Language-Guided Style Transfer Using a Vector-Quantized Tokenizer of a Large-Scale Generative Model [64.26721402514957]
  We propose StylerDALLE, a style transfer method that uses natural language to describe abstract art styles. Specifically, we formulate the language-guided style transfer task as a non-autoregressive token sequence translation. To incorporate style information, we propose a Reinforcement Learning strategy with CLIP-based language supervision.
  arXiv  Detail & Related papers  (2023-03-16T12:44:44Z)
• Generative Artisan: A Semantic-Aware and Controllable CLIPstyler [0.0]
  We present a novel framework that uses a pre-trained CLIP text-image embedding model and guides image style transfer through an FCN semantic segmentation network. Specifically, we solve the portrait over-styling problem for both selfies and real-world landscape with human subjects photos.
  arXiv  Detail & Related papers  (2022-07-23T20:26:47Z)
• Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning [84.8813842101747]
  Contrastive Arbitrary Style Transfer (CAST) is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer.
  arXiv  Detail & Related papers  (2022-05-19T13:11:24Z)
• Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer [103.54337984566877]
  Recent studies on StyleGAN show high performance on artistic portrait generation by transfer learning with limited data. We introduce a novel DualStyleGAN with flexible control of dual styles of the original face domain and the extended artistic portrait domain. Experiments demonstrate the superiority of DualStyleGAN over state-of-the-art methods in high-quality portrait style transfer and flexible style control.
  arXiv  Detail & Related papers  (2022-03-24T17:57:11Z)
• DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN [30.465955123686335]
  We introduce SC-StyleGAN, which injects spatial constraints to the original StyleGAN generation process. Based on SC-StyleGAN, we present DrawingInStyles, a novel drawing interface for non-professional users to easily produce high-quality, photo-realistic face images.
  arXiv  Detail & Related papers  (2022-03-05T14:54:07Z)
• StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation [20.14727435894964]
  We present a caricature generation framework based on shape and style manipulation using StyleGAN. Our framework, dubbed StyleCariGAN, automatically creates a realistic and detailed caricature from an input photo.
  arXiv  Detail & Related papers  (2021-07-09T09:49:31Z)
• StyleMeUp: Towards Style-Agnostic Sketch-Based Image Retrieval [119.03470556503942]
  Crossmodal matching problem is typically solved by learning a joint embedding space where semantic content shared between photo and sketch modalities are preserved. An effective model needs to explicitly account for this style diversity, crucially, to unseen user styles. Our model can not only disentangle the cross-modal shared semantic content, but can adapt the disentanglement to any unseen user style as well, making the model truly agnostic.
  arXiv  Detail & Related papers  (2021-03-29T15:44:19Z)
• Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches [133.01690754567252]
  Sketch-based image editing aims to synthesize and modify photos based on the structural information provided by the human-drawn sketches. Deep Plastic Surgery is a novel, robust and controllable image editing framework that allows users to interactively edit images using hand-drawn sketch inputs.
  arXiv  Detail & Related papers  (2020-01-09T08:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.