Learning Spatially Decoupled Color Representations for Facial Image Colorization

• URL: http://arxiv.org/abs/2412.07203v1
• Date: Tue, 10 Dec 2024 05:48:17 GMT
• Title: Learning Spatially Decoupled Color Representations for Facial Image Colorization
• Authors: Hangyan Zhu, Ming Liu, Chao Zhou, Zifei Yan, Kuanquan Wang, Wangmeng Zuo,
• Abstract summary: We present a novel facial image colorization framework dubbed FCNet. We learn a decoupled color representation for each face component (e.g., lips, skin, eyes, and hair) under the guidance of face parsing maps. FCNet can be naturally applied to facial image colorization with single or multiple reference images.
• Score: 46.8688389538561
• License:
• Abstract: Image colorization methods have shown prominent performance on natural images. However, since humans are more sensitive to faces, existing methods are insufficient to meet the demands when applied to facial images, typically showing unnatural and uneven colorization results. In this paper, we investigate the facial image colorization task and find that the problems with facial images can be attributed to an insufficient understanding of facial components. As a remedy, by introducing facial component priors, we present a novel facial image colorization framework dubbed FCNet. Specifically, we learn a decoupled color representation for each face component (e.g., lips, skin, eyes, and hair) under the guidance of face parsing maps. A chromatic and spatial augmentation strategy is presented to facilitate the learning procedure, which requires only grayscale and color facial image pairs. After training, the presented FCNet can be naturally applied to facial image colorization with single or multiple reference images. To expand the application paradigms to scenarios with no reference images, we further train two alternative modules, which predict the color representations from the grayscale input or a random seed, respectively. Extensive experiments show that our method can perform favorably against existing methods in various application scenarios (i.e., no-, single-, and multi-reference facial image colorization). The source code and pre-trained models will be publicly available.

Related papers

• Single Image, Any Face: Generalisable 3D Face Generation [59.9369171926757]
  We propose a novel model, Gen3D-Face, which generates 3D human faces with unconstrained single image input. To the best of our knowledge, this is the first attempt and benchmark for creating photorealistic 3D human face avatars from single images.
  arXiv  Detail & Related papers  (2024-09-25T14:56:37Z)
• Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation [66.53435569574135]
  Existing facial expression recognition methods typically fine-tune a pre-trained visual encoder using discrete labels. We observe that the rich knowledge in text embeddings, generated by vision-language models, is a promising alternative for learning discriminative facial expression representations. We propose a novel knowledge-enhanced FER method with an emotional-to-neutral transformation.
  arXiv  Detail & Related papers  (2024-09-13T07:28:57Z)
• FlashFace: Human Image Personalization with High-fidelity Identity Preservation [59.76645602354481]
  FlashFace allows users to easily personalize their own photos by providing one or a few reference face images and a text prompt. Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following.
  arXiv  Detail & Related papers  (2024-03-25T17:59:57Z)
• Self-Supervised Facial Representation Learning with Facial Region Awareness [13.06996608324306]
  Self-supervised pre-training has been proven to be effective in learning transferable representations that benefit various visual tasks. Recent efforts toward this goal are limited to treating each face image as a whole. We propose a novel self-supervised facial representation learning framework to learn consistent global and local facial representations.
  arXiv  Detail & Related papers  (2024-03-04T15:48:56Z)
• Neural Implicit Morphing of Face Images [0.7643309077806446]
  Face morphing is a problem in computer graphics with numerous artistic and forensic applications. This task consists of a warping for feature alignment and a blending for a seamless transition between the warped images. We propose to leverage coord-based neural networks to represent such warpings and blendings of face images.
  arXiv  Detail & Related papers  (2023-08-26T14:12:19Z)
• A survey on facial image deblurring [3.6775758132528877]
  When the facial image is blurred, it has a great impact on high-level vision tasks such as face recognition. This paper surveys and summarizes recently published methods for facial image deblurring, most of which are based on deep learning. We show the performance of classical methods on datasets and metrics and give a brief discussion on the differences of model-based and learning-based methods.
  arXiv  Detail & Related papers  (2023-02-10T02:24:56Z)
• Diverse facial inpainting guided by exemplars [8.360536784609309]
  This paper introduces EXE-GAN, a novel diverse and interactive facial inpainting framework. The proposed facial inpainting is achieved based on generative adversarial networks by leveraging the global style of input image, the style, and exemplar style of image. A variety of experimental results and comparisons on public CelebA-HQ and FFHQ datasets are presented to demonstrate the superiority of the proposed method.
  arXiv  Detail & Related papers  (2022-02-13T16:29:45Z)
• FaceEraser: Removing Facial Parts for Augmented Reality [10.575917056215289]
  Our task is to remove all facial parts and then impose visual elements onto the blank'' face for augmented reality. We propose a novel data generation technique to produce paired training data that well mimic the blank'' faces. Our method has been integrated into commercial products and its effectiveness has been verified with unconstrained user inputs.
  arXiv  Detail & Related papers  (2021-09-22T14:30:12Z)
• Pro-UIGAN: Progressive Face Hallucination from Occluded Thumbnails [53.080403912727604]
  We propose a multi-stage Progressive Upsampling and Inpainting Generative Adversarial Network, dubbed Pro-UIGAN. It exploits facial geometry priors to replenish and upsample (8*) the occluded and tiny faces. Pro-UIGAN achieves visually pleasing HR faces, reaching superior performance in downstream tasks.
  arXiv  Detail & Related papers  (2021-08-02T02:29:24Z)
• Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection [65.92058628082322]
  Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
  arXiv  Detail & Related papers  (2021-06-15T03:10:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.