Scalable and Realistic Virtual Try-on Application for Foundation Makeup with Kubelka-Munk Theory

• URL: http://arxiv.org/abs/2507.07333v1
• Date: Wed, 09 Jul 2025 23:19:28 GMT
• Title: Scalable and Realistic Virtual Try-on Application for Foundation Makeup with Kubelka-Munk Theory
• Authors: Hui Pang, Sunil Hadap, Violetta Shevchenko, Rahul Suresh, Amin Banitalebi-Dehkordi,
• Abstract summary: A critical technical challenge in foundation VTO applications is the accurate synthesis of foundation-skin tone color blending.<n>We propose a novel method to approximate well-established Kubelka-Munk (KM) theory for faster image synthesis.<n>We build a scalable end-to-end framework for realistic foundation makeup VTO solely depending on the product information available on e-commerce sites.
• Score: 8.865360031361705
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Augmented reality is revolutionizing beauty industry with virtual try-on (VTO) applications, which empowers users to try a wide variety of products using their phones without the hassle of physically putting on real products. A critical technical challenge in foundation VTO applications is the accurate synthesis of foundation-skin tone color blending while maintaining the scalability of the method across diverse product ranges. In this work, we propose a novel method to approximate well-established Kubelka-Munk (KM) theory for faster image synthesis while preserving foundation-skin tone color blending realism. Additionally, we build a scalable end-to-end framework for realistic foundation makeup VTO solely depending on the product information available on e-commerce sites. We validate our method using real-world makeup images, demonstrating that our framework outperforms other techniques.

Related papers

• DreamMakeup: Face Makeup Customization using Latent Diffusion Models [42.98379243094055]
  We introduce DreamMakup, a novel training-free Diffusion model based Makeup Customization method.<n>Our model demonstrates notable improvements over existing GAN-based and recent diffusion-based frameworks.
  arXiv  Detail & Related papers  (2025-10-13T02:29:23Z)
• Clothing agnostic Pre-inpainting Virtual Try-ON [1.7764619921640012]
  CaP-VTON has improved the naturalness and consistency of whole-body clothing syn-thesis.<n> CaP-VTON recorded 92.5%, which is 15.4% better than Leffa in short-sleeved synthesis accuracy.
  arXiv  Detail & Related papers  (2025-09-22T11:58:20Z)
• AvatarMakeup: Realistic Makeup Transfer for 3D Animatable Head Avatars [89.31582684550723]
  AvatarMakeup achieves state-of-the-art makeup transfer quality and consistency throughout animation.<n>Coherent Duplication optimize a global UV map by recoding the averaged facial attributes among the generated makeup images.<n>Experiments demonstrate that AvatarMakeup achieves state-of-the-art makeup transfer quality and consistency throughout animation.
  arXiv  Detail & Related papers  (2025-07-03T08:26:57Z)
• AnomalyPainter: Vision-Language-Diffusion Synergy for Zero-Shot Realistic and Diverse Industrial Anomaly Synthesis [52.081638586098]
  AnomalyPainter is a framework that synergizes Vision Language Large Model, Latent Diffusion Model, and texture library Tex-9K.<n> Tex-9K is a professional texture library containing 75 categories and 8,792 texture assets crafted for diverse anomaly synthesis.<n>Extensive experiments show that AnomalyPainter outperforms existing methods in realism, diversity, and generalization.
  arXiv  Detail & Related papers  (2025-03-10T12:35:35Z)
• Materialist: Physically Based Editing Using Single-Image Inverse Rendering [47.85234717907478]
  Materialist is a method combining a learning-based approach with physically based progressive differentiable rendering.<n>Our approach enables a range of applications, including material editing, object insertion, and relighting.<n> Experiments demonstrate strong performance across synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2025-01-07T11:52:01Z)
• BeautyBank: Encoding Facial Makeup in Latent Space [2.113770213797994]
  We propose BeautyBank, a novel makeup encoder that disentangles pattern features of bare and makeup faces. Our method encodes makeup features into a high-dimensional space, preserving essential details necessary for makeup reconstruction. We also propose a Progressive Makeup Tuning (PMT) strategy, specifically designed to enhance the preservation of detailed makeup features.
  arXiv  Detail & Related papers  (2024-11-18T01:52:31Z)
• Self-Supervised Vision Transformer for Enhanced Virtual Clothes Try-On [21.422611451978863]
  We introduce an innovative approach for virtual clothes try-on, utilizing a self-supervised Vision Transformer (ViT) and a diffusion model. Our method emphasizes detail enhancement by contrasting local clothing image embeddings, generated by ViT, with their global counterparts. The experimental results showcase substantial advancements in the realism and precision of details in virtual try-on experiences.
  arXiv  Detail & Related papers  (2024-06-15T07:46:22Z)
• REFRAME: Reflective Surface Real-Time Rendering for Mobile Devices [51.983541908241726]
  This work tackles the challenging task of achieving real-time novel view synthesis for reflective surfaces across various scenes. Existing real-time rendering methods, especially those based on meshes, often have subpar performance in modeling surfaces with rich view-dependent appearances. We decompose the color into diffuse and specular, and model the specular color in the reflected direction based on a neural environment map.
  arXiv  Detail & Related papers  (2024-03-25T07:07:50Z)
• Stable-Makeup: When Real-World Makeup Transfer Meets Diffusion Model [15.380297080210559]
  Current makeup transfer methods are limited to simple makeup styles, making them difficult to apply in real-world scenarios.<n>We introduce Stable-Makeup, a novel diffusion-based makeup transfer method capable of robustly transferring a wide range of real-world makeup.
  arXiv  Detail & Related papers  (2024-03-12T15:53:14Z)
• EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction [53.28220984270622]
  3D reconstruction methods should generate high-fidelity results with 3D consistency in real-time.<n>Our method can reconstruct high-quality appearance and accurate mesh on both synthetic and real-world datasets.<n>Our method can be trained in just 1-2 hours using a single GPU and run on mobile devices at over 40 FPS (Frames Per Second)
  arXiv  Detail & Related papers  (2023-11-16T11:30:56Z)
• FLARE: Fast Learning of Animatable and Relightable Mesh Avatars [64.48254296523977]
  Our goal is to efficiently learn personalized animatable 3D head avatars from videos that are geometrically accurate, realistic, relightable, and compatible with current rendering systems. We introduce FLARE, a technique that enables the creation of animatable and relightable avatars from a single monocular video.
  arXiv  Detail & Related papers  (2023-10-26T16:13:00Z)
• LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On [35.4056826207203]
  This work introduces LaDI-VTON, the first Latent Diffusion textual Inversion-enhanced model for the Virtual Try-ON task. The proposed architecture relies on a latent diffusion model extended with a novel additional autoencoder module. We show that our approach outperforms the competitors by a consistent margin, achieving a significant milestone for the task.
  arXiv  Detail & Related papers  (2023-05-22T21:38:06Z)
• Real-time Virtual-Try-On from a Single Example Image through Deep Inverse Graphics and Learned Differentiable Renderers [13.894134334543363]
  We propose a novel framework based on deep learning to build a real-time inverse graphics encoder. Our imitator is a generative network that learns to accurately reproduce the behavior of a given non-differentiable image. Our framework enables novel applications where consumers can virtually try-on a novel unknown product from an inspirational reference image.
  arXiv  Detail & Related papers  (2022-05-12T18:44:00Z)
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs [119.23922747230193]
  We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2021-12-11T11:36:30Z)
• Deep Graphics Encoder for Real-Time Video Makeup Synthesis from Example [11.377850833795494]
  We introduce an inverse computer graphics method for automatic makeup synthesis from a reference image. This method can be used by artists to automatically create realistic virtual cosmetics image samples, or by consumers to virtually try-on a makeup extracted from their favorite reference image.
  arXiv  Detail & Related papers  (2021-05-12T08:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.