Advancing Facial Stylization through Semantic Preservation Constraint and Pseudo-Paired Supervision

• URL: http://arxiv.org/abs/2506.22022v2
• Date: Mon, 30 Jun 2025 07:02:04 GMT
• Title: Advancing Facial Stylization through Semantic Preservation Constraint and Pseudo-Paired Supervision
• Authors: Zhanyi Lu, Yue Zhou,
• Abstract summary: We argue that these issues stem from neglecting semantic shift of the generator during stylization.<n>We propose a facial stylization method that integrates semantic preservation constraint and pseudo-paired supervision.<n>Building upon our facial stylization framework, we achieve more flexible multimodal and reference-guided stylization.
• Score: 3.4228848885035092
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial stylization aims to transform facial images into appealing, high-quality stylized portraits, with the critical challenge of accurately learning the target style while maintaining content consistency with the original image. Although previous StyleGAN-based methods have made significant advancements, the generated results still suffer from artifacts or insufficient fidelity to the source image. We argue that these issues stem from neglecting semantic shift of the generator during stylization. Therefore, we propose a facial stylization method that integrates semantic preservation constraint and pseudo-paired supervision to enhance the content correspondence and improve the stylization effect. Additionally, we develop a methodology for creating multi-level pseudo-paired datasets to implement supervisory constraint. Furthermore, building upon our facial stylization framework, we achieve more flexible multimodal and reference-guided stylization without complex network architecture designs or additional training. Experimental results demonstrate that our approach produces high-fidelity, aesthetically pleasing facial style transfer that surpasses previous methods.

Related papers

• PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium [55.72249032433108]
  PersonaMagic is a stage-regulated generative technique designed for high-fidelity face customization.<n>Our method learns a series of embeddings within a specific timestep interval to capture face concepts.<n>Tests confirm the superiority of PersonaMagic over state-of-the-art methods in both qualitative and quantitative evaluations.
  arXiv  Detail & Related papers  (2024-12-20T08:41:25Z)
• Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models [69.50286698375386]
  We propose a novel approach that better harnesses diffusion models for face-swapping. We introduce a mask shuffling technique during inpainting training, which allows us to create a so-called universal model for swapping. Ours is a relatively unified approach and so it is resilient to errors in other off-the-shelf models.
  arXiv  Detail & Related papers  (2024-09-11T13:43:53Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• High-Fidelity Face Swapping with Style Blending [16.024260677867076]
  We propose an innovative end-to-end framework for high-fidelity face swapping. First, we introduce a StyleGAN-based facial attributes encoder that extracts essential features from faces and inverts them into a latent style code. Second, we introduce an attention-based style blending module to effectively transfer Face IDs from source to target.
  arXiv  Detail & Related papers  (2023-12-17T23:22:37Z)
• Portrait Diffusion: Training-free Face Stylization with Chain-of-Painting [64.43760427752532]
  Face stylization refers to the transformation of a face into a specific portrait style. Current methods require the use of example-based adaptation approaches to fine-tune pre-trained generative models. This paper proposes a training-free face stylization framework, named Portrait Diffusion.
  arXiv  Detail & Related papers  (2023-12-03T06:48:35Z)
• Face Cartoonisation For Various Poses Using StyleGAN [0.7673339435080445]
  This paper presents an innovative approach to achieve face cartoonisation while preserving the original identity and accommodating various poses. We achieve this by introducing an encoder that captures both pose and identity information from images and generates a corresponding embedding within the StyleGAN latent space. We show by extensive experimentation how our encoder adapts the StyleGAN output to better preserve identity when the objective is cartoonisation.
  arXiv  Detail & Related papers  (2023-09-26T13:10:25Z)
• Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation [75.91455714614966]
  We propose Scenimefy, a novel semi-supervised image-to-image translation framework. Our approach guides the learning with structure-consistent pseudo paired data. A patch-wise contrastive style loss is introduced to improve stylization and fine details.
  arXiv  Detail & Related papers  (2023-08-24T17:59:50Z)
• FISTNet: FusIon of STyle-path generative Networks for Facial Style Transfer [15.308837341075135]
  StyleGAN methods have the tendency of overfitting that results in the introduction of artifacts in the facial images. We propose a FusIon of STyles (FIST) network for facial images that leverages pre-trained multipath style transfer networks.
  arXiv  Detail & Related papers  (2023-07-18T07:20:31Z)
• Multi-Modal Face Stylization with a Generative Prior [27.79677001997915]
  MMFS supports multi-modal face stylization by leveraging the strengths of StyleGAN. We introduce a two-stage training strategy, where we train the encoder in the first stage to align the feature maps with StyleGAN. In the second stage, the entire network is fine-tuned with artistic data for stylized face generation.
  arXiv  Detail & Related papers  (2023-05-29T11:01:31Z)
• BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generation [9.370501805054344]
  We propose BlendGAN for arbitrary stylized face generation. We first train a self-supervised style encoder on the generic artistic dataset to extract the representations of arbitrary styles. In addition, a weighted blending module (WBM) is proposed to blend face and style representations implicitly and control the arbitrary stylization effect.
  arXiv  Detail & Related papers  (2021-10-22T12:00:27Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.