High-Quality Real Time Facial Capture Based on Single Camera
- URL: http://arxiv.org/abs/2111.07556v1
- Date: Mon, 15 Nov 2021 06:42:27 GMT
- Title: High-Quality Real Time Facial Capture Based on Single Camera
- Authors: Hongwei Xu and Leijia Dai and Jianxing Fu and Xiangyuan Wang and
Quanwei Wang
- Abstract summary: We train a convolutional neural network to produce high-quality continuous blendshape weight output from video training.
We demonstrate compelling animation inference in challenging areas such as eyes and lips.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose a real time deep learning framework for video-based facial
expression capture. Our process uses a high-end facial capture pipeline based
on FACEGOOD to capture facial expression. We train a convolutional neural
network to produce high-quality continuous blendshape weight output from video
training. Since this facial capture is fully automated, our system can
drastically reduce the amount of labor involved in the development of modern
narrative-driven video games or films involving realistic digital doubles of
actors and potentially hours of animated dialogue per character. We demonstrate
compelling animation inference in challenging areas such as eyes and lips.
Related papers
- Attention-Based VR Facial Animation with Visual Mouth Camera Guidance
for Immersive Telepresence Avatars [19.70403947793871]
We present a hybrid method that uses both keypoints and direct visual guidance from a mouth camera.
Our method generalizes to unseen operators and requires only a quick enrolment step with capture of two short videos.
We highlight how the facial animation contributed to our victory at the ANA Avatar XPRIZE Finals.
arXiv Detail & Related papers (2023-12-15T12:45:11Z) - Audio-Driven Talking Face Generation with Diverse yet Realistic Facial
Animations [61.65012981435094]
DIRFA is a novel method that can generate talking faces with diverse yet realistic facial animations from the same driving audio.
To accommodate fair variation of plausible facial animations for the same audio, we design a transformer-based probabilistic mapping network.
We show that DIRFA can generate talking faces with realistic facial animations effectively.
arXiv Detail & Related papers (2023-04-18T12:36:15Z) - 4D Agnostic Real-Time Facial Animation Pipeline for Desktop Scenarios [8.274472944075713]
We present a high-precision real-time facial animation pipeline suitable for animators to use on their desktops.
System enables animators to create high-quality facial animations with ease and speed.
Our approach has the potential to revolutionize the way facial animation is done in the entertainment industry.
arXiv Detail & Related papers (2023-04-06T01:32:58Z) - Relightable Neural Video Portrait [36.67623086400362]
Photo-realistic facial video portrait reenactment benefits virtual production and numerous VR/AR experiences.
We present a relightable neural video portrait, a simultaneous relighting and reenactment scheme that transfers the head pose and facial expressions from a source actor to a portrait video of a target actor with arbitrary new backgrounds and lighting conditions.
arXiv Detail & Related papers (2021-07-30T16:20:45Z) - Image-to-Video Generation via 3D Facial Dynamics [78.01476554323179]
We present a versatile model, FaceAnime, for various video generation tasks from still images.
Our model is versatile for various AR/VR and entertainment applications, such as face video and face video prediction.
arXiv Detail & Related papers (2021-05-31T02:30:11Z) - Audio- and Gaze-driven Facial Animation of Codec Avatars [149.0094713268313]
We describe the first approach to animate Codec Avatars in real-time using audio and/or eye tracking.
Our goal is to display expressive conversations between individuals that exhibit important social signals.
arXiv Detail & Related papers (2020-08-11T22:28:48Z) - ReenactNet: Real-time Full Head Reenactment [50.32988828989691]
We propose a head-to-head system capable of fully transferring the human head 3D pose, facial expressions and eye gaze from a source to a target actor.
Our system produces high-fidelity, temporally-smooth and photo-realistic synthetic videos faithfully transferring the human time-varying head attributes from the source to the target actor.
arXiv Detail & Related papers (2020-05-22T00:51:38Z) - DeepFaceFlow: In-the-wild Dense 3D Facial Motion Estimation [56.56575063461169]
DeepFaceFlow is a robust, fast, and highly-accurate framework for the estimation of 3D non-rigid facial flow.
Our framework was trained and tested on two very large-scale facial video datasets.
Given registered pairs of images, our framework generates 3D flow maps at 60 fps.
arXiv Detail & Related papers (2020-05-14T23:56:48Z) - Audio-driven Talking Face Video Generation with Learning-based
Personalized Head Pose [67.31838207805573]
We propose a deep neural network model that takes an audio signal A of a source person and a short video V of a target person as input.
We outputs a synthesized high-quality talking face video with personalized head pose.
Our method can generate high-quality talking face videos with more distinguishing head movement effects than state-of-the-art methods.
arXiv Detail & Related papers (2020-02-24T10:02:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.