FaceChain: A Playground for Human-centric Artificial Intelligence
Generated Content
- URL: http://arxiv.org/abs/2308.14256v2
- Date: Thu, 14 Dec 2023 03:35:18 GMT
- Title: FaceChain: A Playground for Human-centric Artificial Intelligence
Generated Content
- Authors: Yang Liu, Cheng Yu, Lei Shang, Yongyi He, Ziheng Wu, Xingjun Wang,
Chao Xu, Haoyu Xie, Weida Wang, Yuze Zhao, Lin Zhu, Chen Cheng, Weitao Chen,
Yuan Yao, Wenmeng Zhou, Jiaqi Xu, Qiang Wang, Yingda Chen, Xuansong Xie,
Baigui Sun
- Abstract summary: FaceChain is a personalized portrait generation framework that combines a series of customized image-generation model and a rich set of face-related perceptual understanding models.
We inject several SOTA face models into the generation procedure, achieving a more efficient label-tagging, data-processing, and model post-processing compared to previous solutions.
Based on FaceChain, we further develop several applications to build a broader playground for better showing its value, including virtual try on and 2D talking head.
- Score: 36.48960592782015
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent advancement in personalized image generation have unveiled the
intriguing capability of pre-trained text-to-image models on learning identity
information from a collection of portrait images. However, existing solutions
are vulnerable in producing truthful details, and usually suffer from several
defects such as (i) The generated face exhibit its own unique characteristics,
\ie facial shape and facial feature positioning may not resemble key
characteristics of the input, and (ii) The synthesized face may contain warped,
blurred or corrupted regions. In this paper, we present FaceChain, a
personalized portrait generation framework that combines a series of customized
image-generation model and a rich set of face-related perceptual understanding
models (\eg, face detection, deep face embedding extraction, and facial
attribute recognition), to tackle aforementioned challenges and to generate
truthful personalized portraits, with only a handful of portrait images as
input. Concretely, we inject several SOTA face models into the generation
procedure, achieving a more efficient label-tagging, data-processing, and model
post-processing compared to previous solutions, such as DreamBooth
~\cite{ruiz2023dreambooth} , InstantBooth ~\cite{shi2023instantbooth} , or
other LoRA-only approaches ~\cite{hu2021lora} . Besides, based on FaceChain, we
further develop several applications to build a broader playground for better
showing its value, including virtual try on and 2D talking head. We hope it can
grow to serve the burgeoning needs from the communities. Note that this is an
ongoing work that will be consistently refined and improved upon. FaceChain is
open-sourced under Apache-2.0 license at
\url{https://github.com/modelscope/facechain}.
Related papers
- FlashFace: Human Image Personalization with High-fidelity Identity Preservation [59.76645602354481]
FlashFace allows users to easily personalize their own photos by providing one or a few reference face images and a text prompt.
Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following.
arXiv Detail & Related papers (2024-03-25T17:59:57Z) - Arc2Face: A Foundation Model of Human Faces [95.00331107591859]
Arc2Face is an identity-conditioned face foundation model.
It can generate diverse photo-realistic images with an unparalleled degree of face similarity than existing models.
arXiv Detail & Related papers (2024-03-18T10:32:51Z) - VIGFace: Virtual Identity Generation Model for Face Image Synthesis [13.81887339529775]
We propose VIGFace, a novel framework capable of generating synthetic facial images.
It allows for creating virtual facial images without concerns about portrait rights.
It serves as an effective augmentation method by incorporating real existing images.
arXiv Detail & Related papers (2024-03-13T06:11:41Z) - DreamIdentity: Improved Editability for Efficient Face-identity
Preserved Image Generation [69.16517915592063]
We propose a novel face-identity encoder to learn an accurate representation of human faces.
We also propose self-augmented editability learning to enhance the editability of models.
Our methods can generate identity-preserved images under different scenes at a much faster speed.
arXiv Detail & Related papers (2023-07-01T11:01:17Z) - Graph-based Generative Face Anonymisation with Pose Preservation [49.18049578591058]
AnonyGAN is a GAN-based solution for face anonymisation.
It replaces the visual information corresponding to a source identity with a condition identity provided as any single image.
arXiv Detail & Related papers (2021-12-10T12:58:17Z) - FaceEraser: Removing Facial Parts for Augmented Reality [10.575917056215289]
Our task is to remove all facial parts and then impose visual elements onto the blank'' face for augmented reality.
We propose a novel data generation technique to produce paired training data that well mimic the blank'' faces.
Our method has been integrated into commercial products and its effectiveness has been verified with unconstrained user inputs.
arXiv Detail & Related papers (2021-09-22T14:30:12Z) - Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo
Collection [65.92058628082322]
Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions.
This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
arXiv Detail & Related papers (2021-06-15T03:10:17Z) - Image-to-Video Generation via 3D Facial Dynamics [78.01476554323179]
We present a versatile model, FaceAnime, for various video generation tasks from still images.
Our model is versatile for various AR/VR and entertainment applications, such as face video and face video prediction.
arXiv Detail & Related papers (2021-05-31T02:30:11Z) - VariTex: Variational Neural Face Textures [0.0]
VariTex is a method that learns a variational latent feature space of neural face textures.
To generate images of complete human heads, we propose an additive decoder that generates plausible additional details such as hair.
The resulting method can generate geometrically consistent images of novel identities allowing fine-grained control over head pose, face shape, and facial expressions.
arXiv Detail & Related papers (2021-04-13T07:47:53Z) - Realistic Face Reenactment via Self-Supervised Disentangling of Identity
and Pose [23.211318473026243]
We propose a self-supervised hybrid model (DAE-GAN) that learns how to reenact face naturally given large amounts of unlabeled videos.
Our approach combines two deforming autoencoders with the latest advances in the conditional generation.
Experiment results demonstrate the superior quality of reenacted images and the flexibility of transferring facial movements between identities.
arXiv Detail & Related papers (2020-03-29T06:45:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.