From Synthetic to Real: Toward Identity-Consistent Makeup Transfer with Synthetic and Real Data

• URL: http://arxiv.org/abs/2605.07861v1
• Date: Fri, 08 May 2026 15:21:43 GMT
• Title: From Synthetic to Real: Toward Identity-Consistent Makeup Transfer with Synthetic and Real Data
• Abstract summary: Makeup transfer aims to apply the makeup style of a reference portrait to a source portrait while preserving identity and background.<n>Recent diffusion- and flow-based approaches instead exploit synthetic data for supervised training, leading to significant improvements.<n>This paper proposes ConsistentBeauty and RealBeauty, a synthetic-to-real post-training framework.
• Score: 63.40634698361175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Makeup transfer aims to apply the makeup style of a reference portrait to a source portrait while preserving identity and background. Early methods formulate this task as unsupervised image-to-image translation, relying on surrogate objectives and often yielding limited performance. Recent diffusion- and flow-based approaches instead exploit synthetic data for supervised training, leading to significant improvements. However, these methods still face two critical challenges: synthetic supervision frequently fails to faithfully preserve identity, and the domain gap between synthetic and real data limits generalization, resulting in degraded performance in complex real-world scenarios. To address these issues, this paper first proposes ConsistentBeauty, a novel data curation pipeline that ensures makeup fidelity and strict identity consistency within the synthesized data. Second, we propose RealBeauty, a synthetic-to-real post-training framework. Beyond supervised learning on curated synthetic data, we further adapt the model to real-world scenarios through reinforcement learning and design novel verifiable rewards tailored to the makeup transfer task. It allows the model to further benefit from real makeup patterns beyond synthetic supervision. In addition, we establish a new diverse benchmark for makeup transfer, covering a wide range of skin tones, ages, genders, poses, and makeup styles, thereby enabling a more comprehensive evaluation of model performance under diverse real-world conditions. Extensive experiments show that our method achieves state-of-the-art performance on multiple benchmarks and demonstrates clear advantages in identity preservation and performance on complex real-world cases.

Related papers

• Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer [70.44549081321442]
  Makeup transfer applies a reference cosmetic style to a source face while preserving its identity and geometry.<n>Current methods rely on weak priors or synthetic pseudo-targets from large-scale editing models.<n>We propose Anchoring on Reality Makeup Transfer (ART), a two-stage framework with a reality-anchored refinement cycle.
  arXiv  Detail & Related papers  (2026-06-30T03:28:01Z)
• Supervised makeup transfer with a curated dataset: Decoupling identity and makeup features for enhanced transformation [21.71636658071446]
  Diffusion models have shown strong progress in generative tasks, offering a more stable alternative to GAN-based approaches for makeup transfer.<n>Existing methods often suffer from limited datasets, poor disentanglement between identity and makeup features, and weak controllability.<n>We construct a curated high-quality dataset using a train-generate-filter-retrain strategy that combines synthetic, realistic, and filtered samples to improve diversity and fidelity.<n>Third, we propose a text-guided mechanism that allows fine-grained and region-specific control, enabling users to modify eyes, lips, or face makeup with natural language prompts.
  arXiv  Detail & Related papers  (2026-01-31T13:46:38Z)
• Understanding the Influence of Synthetic Data for Text Embedders [52.04771455432998]
  We first reproduce and publicly release the synthetic data proposed by Wang et al.<n>We critically examine where exactly synthetic data improves model generalization.<n>Our findings highlight the limitations of current synthetic data approaches for building general-purpose embedders.
  arXiv  Detail & Related papers  (2025-09-07T19:28:52Z)
• Towards High-Fidelity, Identity-Preserving Real-Time Makeup Transfer: Decoupling Style Generation [10.030819778997836]
  We present a novel framework for real-time virtual makeup try-on.<n>It achieves high-fidelity, identity-preserving cosmetic transfer with robust temporal consistency.
  arXiv  Detail & Related papers  (2025-09-02T15:52:56Z)
• SynFER: Towards Boosting Facial Expression Recognition with Synthetic Data [78.70620682374624]
  We introduce SynFER, a novel framework for synthesizing facial expression image data based on high-level textual descriptions.<n>To ensure the quality and reliability of the synthetic data, we propose a semantic guidance technique and a pseudo-label generator.<n>Results validate the efficacy of our approach and the synthetic data.
  arXiv  Detail & Related papers  (2024-10-13T14:58:21Z)
• Is Synthetic Image Useful for Transfer Learning? An Investigation into Data Generation, Volume, and Utilization [62.157627519792946]
  We introduce a novel framework called bridged transfer, which initially employs synthetic images for fine-tuning a pre-trained model to improve its transferability. We propose dataset style inversion strategy to improve the stylistic alignment between synthetic and real images. Our proposed methods are evaluated across 10 different datasets and 5 distinct models, demonstrating consistent improvements.
  arXiv  Detail & Related papers  (2024-03-28T22:25:05Z)
• ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning [102.46382882098847]
  We first investigate the effects of synthetic data in synthetic-to-real novel view synthesis. We propose to introduce geometry-aware contrastive learning to learn multi-view consistent features with geometric constraints. Our method can render images with higher quality and better fine-grained details, outperforming existing generalizable novel view synthesis methods in terms of PSNR, SSIM, and LPIPS.
  arXiv  Detail & Related papers  (2023-03-20T12:06:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.