Turn That Frown Upside Down: FaceID Customization via Cross-Training Data

• URL: http://arxiv.org/abs/2501.15407v1
• Date: Sun, 26 Jan 2025 05:27:38 GMT
• Title: Turn That Frown Upside Down: FaceID Customization via Cross-Training Data
• Authors: Shuhe Wang, Xiaoya Li, Xiaofei Sun, Guoyin Wang, Tianwei Zhang, Jiwei Li, Eduard Hovy,
• Abstract summary: CrossFaceID is the first large-scale, high-quality, and publicly available dataset designed to improve the facial modification capabilities of FaceID customization models. It consists of 40,000 text-image pairs from approximately 2,000 persons, with each person represented by around 20 images showcasing diverse facial attributes. During the training stage, a specific face of a person is used as input, and the FaceID customization model is forced to generate another image of the same person but with altered facial features. Experiments show that models fine-tuned on the CrossFaceID dataset its performance in preserving FaceID fidelity while significantly improving its
• Score: 49.51940625552275
• License:
• Abstract: Existing face identity (FaceID) customization methods perform well but are limited to generating identical faces as the input, while in real-world applications, users often desire images of the same person but with variations, such as different expressions (e.g., smiling, angry) or angles (e.g., side profile). This limitation arises from the lack of datasets with controlled input-output facial variations, restricting models' ability to learn effective modifications. To address this issue, we propose CrossFaceID, the first large-scale, high-quality, and publicly available dataset specifically designed to improve the facial modification capabilities of FaceID customization models. Specifically, CrossFaceID consists of 40,000 text-image pairs from approximately 2,000 persons, with each person represented by around 20 images showcasing diverse facial attributes such as poses, expressions, angles, and adornments. During the training stage, a specific face of a person is used as input, and the FaceID customization model is forced to generate another image of the same person but with altered facial features. This allows the FaceID customization model to acquire the ability to personalize and modify known facial features during the inference stage. Experiments show that models fine-tuned on the CrossFaceID dataset retain its performance in preserving FaceID fidelity while significantly improving its face customization capabilities. To facilitate further advancements in the FaceID customization field, our code, constructed datasets, and trained models are fully available to the public.

Related papers

• Face Anonymization Made Simple [44.24233169815565]
  Current face anonymization techniques often depend on identity loss calculated by face recognition models, which can be inaccurate and unreliable. In contrast, our approach uses diffusion models with only a reconstruction loss, eliminating the need for facial landmarks or masks. Our model achieves state-of-the-art performance in three key areas: identity anonymization, facial preservation, and image quality.
  arXiv  Detail & Related papers  (2024-11-01T17:45:21Z)
• FaceChain-FACT: Face Adapter with Decoupled Training for Identity-preserved Personalization [24.600720169589334]
  adapter-based method obtains the ability to customize and generate portraits by text-to-image training on facial data. There is often a significant performance decrease in test following ability, controllability, and diversity of generated faces compared to the base model. We propose the Face Adapter with deCoupled Training (FACT) framework, focusing on both model architecture and training strategy.
  arXiv  Detail & Related papers  (2024-10-16T07:25:24Z)
• FlashFace: Human Image Personalization with High-fidelity Identity Preservation [59.76645602354481]
  FlashFace allows users to easily personalize their own photos by providing one or a few reference face images and a text prompt. Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following.
  arXiv  Detail & Related papers  (2024-03-25T17:59:57Z)
• StableIdentity: Inserting Anybody into Anywhere at First Sight [57.99693188913382]
  We propose StableIdentity, which allows identity-consistent recontextualization with just one face image. We are the first to directly inject the identity learned from a single image into video/3D generation without finetuning.
  arXiv  Detail & Related papers  (2024-01-29T09:06:15Z)
• FaceChain: A Playground for Human-centric Artificial Intelligence Generated Content [36.48960592782015]
  FaceChain is a personalized portrait generation framework that combines a series of customized image-generation model and a rich set of face-related perceptual understanding models. We inject several SOTA face models into the generation procedure, achieving a more efficient label-tagging, data-processing, and model post-processing compared to previous solutions. Based on FaceChain, we further develop several applications to build a broader playground for better showing its value, including virtual try on and 2D talking head.
  arXiv  Detail & Related papers  (2023-08-28T02:20:44Z)
• DreamIdentity: Improved Editability for Efficient Face-identity Preserved Image Generation [69.16517915592063]
  We propose a novel face-identity encoder to learn an accurate representation of human faces. We also propose self-augmented editability learning to enhance the editability of models. Our methods can generate identity-preserved images under different scenes at a much faster speed.
  arXiv  Detail & Related papers  (2023-07-01T11:01:17Z)
• A Systematical Solution for Face De-identification [6.244117712209321]
  In different tasks, people have various requirements for face de-identification (De-ID) We propose a systematical solution compatible for these De-ID operations. Our method can flexibly de-identify the face data in various ways and the processed images have high image quality.
  arXiv  Detail & Related papers  (2021-07-19T02:02:51Z)
• FedFace: Collaborative Learning of Face Recognition Model [66.84737075622421]
  FedFace is a framework for collaborative learning of face recognition models. It learns an accurate and generalizable face recognition model where the face images stored at each client are neither shared with other clients nor the central host. Our code and pre-trained models will be publicly available.
  arXiv  Detail & Related papers  (2021-04-07T09:25:32Z)
• DotFAN: A Domain-transferred Face Augmentation Network for Pose and Illumination Invariant Face Recognition [94.96686189033869]
  We propose a 3D model-assisted domain-transferred face augmentation network (DotFAN) DotFAN can generate a series of variants of an input face based on the knowledge distilled from existing rich face datasets collected from other domains. Experiments show that DotFAN is beneficial for augmenting small face datasets to improve their within-class diversity.
  arXiv  Detail & Related papers  (2020-02-23T08:16:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.