FAGhead: Fully Animate Gaussian Head from Monocular Videos
- URL: http://arxiv.org/abs/2406.19070v2
- Date: Fri, 28 Jun 2024 06:47:10 GMT
- Title: FAGhead: Fully Animate Gaussian Head from Monocular Videos
- Authors: Yixin Xuan, Xinyang Li, Gongxin Yao, Shiwei Zhou, Donghui Sun, Xiaoxin Chen, Yu Pan,
- Abstract summary: FAGhead is a method that enables fully controllable human portraits from monocular videos.
We explicit the traditional 3D morphable meshes (3DMM) and optimize the neutral 3D Gaussians to reconstruct with complex expressions.
To effectively manage the edges of avatars, we introduced the alpha rendering to supervise the alpha value of each pixel.
- Score: 2.9979421496374683
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: High-fidelity reconstruction of 3D human avatars has a wild application in visual reality. In this paper, we introduce FAGhead, a method that enables fully controllable human portraits from monocular videos. We explicit the traditional 3D morphable meshes (3DMM) and optimize the neutral 3D Gaussians to reconstruct with complex expressions. Furthermore, we employ a novel Point-based Learnable Representation Field (PLRF) with learnable Gaussian point positions to enhance reconstruction performance. Meanwhile, to effectively manage the edges of avatars, we introduced the alpha rendering to supervise the alpha value of each pixel. Extensive experimental results on the open-source datasets and our capturing datasets demonstrate that our approach is able to generate high-fidelity 3D head avatars and fully control the expression and pose of the virtual avatars, which is outperforming than existing works.
Related papers
- 3D Gaussian Parametric Head Model [40.62136721707944]
This paper introduces a novel approach, 3D Gaussian Parametric Head Model, which employs 3D Gaussians to accurately represent the complexities of the human head.
It enables seamless face portrait and the reconstruction of detailed head avatars from a single image.
Our method achieves high-quality, photo-realistic rendering with real-time efficiency, making it a valuable contribution to the field of parametric head models.
arXiv Detail & Related papers (2024-07-21T06:03:11Z) - NPGA: Neural Parametric Gaussian Avatars [46.52887358194364]
We propose a data-driven approach to create high-fidelity controllable avatars from multi-view video recordings.
We build our method around 3D Gaussian Splatting for its highly efficient rendering.
We evaluate our method on the public NeRSemble dataset, demonstrating that NPGA significantly outperforms the previous state-of-the-art avatars.
arXiv Detail & Related papers (2024-05-29T17:58:09Z) - GVA: Reconstructing Vivid 3D Gaussian Avatars from Monocular Videos [56.40776739573832]
We present a novel method that facilitates the creation of vivid 3D Gaussian avatars from monocular video inputs (GVA)
Our innovation lies in addressing the intricate challenges of delivering high-fidelity human body reconstructions.
We introduce a pose refinement technique to improve hand and foot pose accuracy by aligning normal maps and silhouettes.
arXiv Detail & Related papers (2024-02-26T14:40:15Z) - Deformable 3D Gaussian Splatting for Animatable Human Avatars [50.61374254699761]
We propose a fully explicit approach to construct a digital avatar from as little as a single monocular sequence.
ParDy-Human constitutes an explicit model for realistic dynamic human avatars which requires significantly fewer training views and images.
Our avatars learning is free of additional annotations such as Splat masks and can be trained with variable backgrounds while inferring full-resolution images efficiently even on consumer hardware.
arXiv Detail & Related papers (2023-12-22T20:56:46Z) - MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar [44.125711148560605]
MonoGaussianAvatar is a novel approach that harnesses 3D Gaussian point representation and a Gaussian deformation field to learn explicit head avatars from monocular portrait videos.
Experiments demonstrate the superior performance of our method, which achieves state-of-the-art results among previous methods.
arXiv Detail & Related papers (2023-12-07T18:59:31Z) - Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and
Editing [53.05069432989608]
We present a novel framework for generating 3D human heads with remarkable flexibility.
Our method facilitates the creation of diverse and realistic 3D human heads with fine-grained editing over facial features and expressions.
arXiv Detail & Related papers (2023-12-05T19:05:58Z) - Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians [41.86540576028268]
We propose controllable 3D Gaussian Head Avatars for lightweight sparse-view setups.
We show our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution even under exaggerated expressions.
arXiv Detail & Related papers (2023-12-05T11:01:44Z) - GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians [51.46168990249278]
We present an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video.
GustafAvatar is validated on both the public dataset and our collected dataset.
arXiv Detail & Related papers (2023-12-04T18:55:45Z) - Learning Personalized High Quality Volumetric Head Avatars from
Monocular RGB Videos [47.94545609011594]
We propose a method to learn a high-quality implicit 3D head avatar from a monocular RGB video captured in the wild.
Our hybrid pipeline combines the geometry prior and dynamic tracking of a 3DMM with a neural radiance field to achieve fine-grained control and photorealism.
arXiv Detail & Related papers (2023-04-04T01:10:04Z) - DRaCoN -- Differentiable Rasterization Conditioned Neural Radiance
Fields for Articulated Avatars [92.37436369781692]
We present DRaCoN, a framework for learning full-body volumetric avatars.
It exploits the advantages of both the 2D and 3D neural rendering techniques.
Experiments on the challenging ZJU-MoCap and Human3.6M datasets indicate that DRaCoN outperforms state-of-the-art methods.
arXiv Detail & Related papers (2022-03-29T17:59:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.