DreamMakeup: Face Makeup Customization using Latent Diffusion Models

• URL: http://arxiv.org/abs/2510.10918v1
• Date: Mon, 13 Oct 2025 02:29:23 GMT
• Title: DreamMakeup: Face Makeup Customization using Latent Diffusion Models
• Authors: Geon Yeong Park, Inhwa Han, Serin Yang, Yeobin Hong, Seongmin Jeong, Heechan Jeon, Myeongjin Goh, Sung Won Yi, Jin Nam, Jong Chul Ye,
• Abstract summary: We introduce DreamMakup, a novel training-free Diffusion model based Makeup Customization method.<n>Our model demonstrates notable improvements over existing GAN-based and recent diffusion-based frameworks.
• Score: 42.98379243094055
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The exponential growth of the global makeup market has paralleled advancements in virtual makeup simulation technology. Despite the progress led by GANs, their application still encounters significant challenges, including training instability and limited customization capabilities. Addressing these challenges, we introduce DreamMakup - a novel training-free Diffusion model based Makeup Customization method, leveraging the inherent advantages of diffusion models for superior controllability and precise real-image editing. DreamMakeup employs early-stopped DDIM inversion to preserve the facial structure and identity while enabling extensive customization through various conditioning inputs such as reference images, specific RGB colors, and textual descriptions. Our model demonstrates notable improvements over existing GAN-based and recent diffusion-based frameworks - improved customization, color-matching capabilities, identity preservation and compatibility with textual descriptions or LLMs with affordable computational costs.

Related papers

• Supervised makeup transfer with a curated dataset: Decoupling identity and makeup features for enhanced transformation [21.71636658071446]
  Diffusion models have shown strong progress in generative tasks, offering a more stable alternative to GAN-based approaches for makeup transfer.<n>Existing methods often suffer from limited datasets, poor disentanglement between identity and makeup features, and weak controllability.<n>We construct a curated high-quality dataset using a train-generate-filter-retrain strategy that combines synthetic, realistic, and filtered samples to improve diversity and fidelity.<n>Third, we propose a text-guided mechanism that allows fine-grained and region-specific control, enabling users to modify eyes, lips, or face makeup with natural language prompts.
  arXiv  Detail & Related papers  (2026-01-31T13:46:38Z)
• Towards High-Fidelity, Identity-Preserving Real-Time Makeup Transfer: Decoupling Style Generation [10.030819778997836]
  We present a novel framework for real-time virtual makeup try-on.<n>It achieves high-fidelity, identity-preserving cosmetic transfer with robust temporal consistency.
  arXiv  Detail & Related papers  (2025-09-02T15:52:56Z)
• MADI: Masking-Augmented Diffusion with Inference-Time Scaling for Visual Editing [41.6713004141353]
  Masking-Augmented Diffusion with Inference-Time Scaling (MADI) is a framework that improves the editability, compositionality and controllability of diffusion models.<n>First, we introduce Masking-Augmented gaussian Diffusion (MAgD), a novel training strategy with dual corruption process.<n>Second, we introduce an inference-time capacity scaling mechanism based on Pause Tokens, which act as special placeholders inserted into the prompt for increasing computational capacity at inference time.
  arXiv  Detail & Related papers  (2025-07-16T20:56:57Z)
• Scalable and Realistic Virtual Try-on Application for Foundation Makeup with Kubelka-Munk Theory [8.865360031361705]
  A critical technical challenge in foundation VTO applications is the accurate synthesis of foundation-skin tone color blending.<n>We propose a novel method to approximate well-established Kubelka-Munk (KM) theory for faster image synthesis.<n>We build a scalable end-to-end framework for realistic foundation makeup VTO solely depending on the product information available on e-commerce sites.
  arXiv  Detail & Related papers  (2025-07-09T23:19:28Z)
• AvatarMakeup: Realistic Makeup Transfer for 3D Animatable Head Avatars [89.31582684550723]
  AvatarMakeup achieves state-of-the-art makeup transfer quality and consistency throughout animation.<n>Coherent Duplication optimize a global UV map by recoding the averaged facial attributes among the generated makeup images.<n>Experiments demonstrate that AvatarMakeup achieves state-of-the-art makeup transfer quality and consistency throughout animation.
  arXiv  Detail & Related papers  (2025-07-03T08:26:57Z)
• Multi-focal Conditioned Latent Diffusion for Person Image Synthesis [59.113899155476005]
  The Latent Diffusion Model (LDM) has demonstrated strong capabilities in high-resolution image generation.<n>We propose a Multi-focal Conditioned Latent Diffusion (MCLD) method to address these limitations.<n>Our approach utilizes a multi-focal condition aggregation module, which effectively integrates facial identity and texture-specific information.
  arXiv  Detail & Related papers  (2025-03-19T20:50:10Z)
• Foundation Cures Personalization: Improving Personalized Models' Prompt Consistency via Hidden Foundation Knowledge [49.36669870661573]
  We propose FreeCure, a framework that improves the prompt consistency of personalization models.<n>We introduce a novel foundation-aware self-attention module, coupled with an inversion-based process to bring well-aligned attribute information to the personalization process.
  arXiv  Detail & Related papers  (2024-11-22T15:21:38Z)
• BeautyBank: Encoding Facial Makeup in Latent Space [2.113770213797994]
  We propose BeautyBank, a novel makeup encoder that disentangles pattern features of bare and makeup faces. Our method encodes makeup features into a high-dimensional space, preserving essential details necessary for makeup reconstruction. We also propose a Progressive Makeup Tuning (PMT) strategy, specifically designed to enhance the preservation of detailed makeup features.
  arXiv  Detail & Related papers  (2024-11-18T01:52:31Z)
• Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models [69.50286698375386]
  We propose a novel approach that better harnesses diffusion models for face-swapping. We introduce a mask shuffling technique during inpainting training, which allows us to create a so-called universal model for swapping. Ours is a relatively unified approach and so it is resilient to errors in other off-the-shelf models.
  arXiv  Detail & Related papers  (2024-09-11T13:43:53Z)
• Fashion Matrix: Editing Photos by Just Talking [66.83502497764698]
  We develop a hierarchical AI system called Fashion Matrix dedicated to editing photos by just talking. Fashion Matrix employs Large Language Models (LLMs) as its foundational support and engages in iterative interactions with users. Visual Foundation Models are leveraged to generate edited images from text prompts and masks, thereby facilitating the automation of fashion editing processes.
  arXiv  Detail & Related papers  (2023-07-25T04:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.