HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting
- URL: http://arxiv.org/abs/2312.02902v1
- Date: Tue, 5 Dec 2023 17:19:22 GMT
- Title: HeadGaS: Real-Time Animatable Head Avatars via 3D Gaussian Splatting
- Authors: Helisa Dhamo, Yinyu Nie, Arthur Moreau, Jifei Song, Richard Shaw,
Yiren Zhou, Eduardo P\'erez-Pellitero
- Abstract summary: We propose HeadGaS, the first model to use 3D Gaussian Splats (3DGS) for 3D head reconstruction and animation.
We demonstrate that HeadGaS delivers state-of-the-art results in real-time inference frame rates, which surpasses baselines by up to 2dB.
- Score: 9.98045783250373
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: 3D head animation has seen major quality and runtime improvements over the
last few years, particularly empowered by the advances in differentiable
rendering and neural radiance fields. Real-time rendering is a highly desirable
goal for real-world applications. We propose HeadGaS, the first model to use 3D
Gaussian Splats (3DGS) for 3D head reconstruction and animation. In this paper
we introduce a hybrid model that extends the explicit representation from 3DGS
with a base of learnable latent features, which can be linearly blended with
low-dimensional parameters from parametric head models to obtain
expression-dependent final color and opacity values. We demonstrate that
HeadGaS delivers state-of-the-art results in real-time inference frame rates,
which surpasses baselines by up to ~2dB, while accelerating rendering speed by
over x10.
Related papers
- iHuman: Instant Animatable Digital Humans From Monocular Videos [16.98924995658091]
We present a fast, simple, yet effective method for creating animatable 3D digital humans from monocular videos.
This work achieves and illustrates the need of accurate 3D mesh-type modelling of the human body.
Our method is faster by an order of magnitude (in terms of training time) than its closest competitor.
arXiv Detail & Related papers (2024-07-15T18:51:51Z) - GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction [52.04103235260539]
We present a diffusion model approach based on Gaussian Splatting representation for 3D object reconstruction from a single view.
The model learns to generate 3D objects represented by sets of GS ellipsoids.
The final reconstructed objects explicitly come with high-quality 3D structure and texture, and can be efficiently rendered in arbitrary views.
arXiv Detail & Related papers (2024-07-05T03:43:08Z) - PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting [59.277480452459315]
We propose a principled spatial sensitivity pruning score that outperforms current approaches.
We also propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model.
Our pipeline increases the average rendering speed of 3D-GS by 2.65$times$ while retaining more salient foreground information.
arXiv Detail & Related papers (2024-06-14T17:53:55Z) - GGHead: Fast and Generalizable 3D Gaussian Heads [48.967905053963385]
3D GANs struggle to scale to generate samples at high resolutions due to their relatively slow train and render speeds.
We propose Generative Gaussian Heads (GGHead), which adopts the recent 3D Gaussian Splatting representation within a 3D GAN framework.
We demonstrate real-time generation and rendering of high-quality 3D-consistent heads at $10242$ resolution for the first time.
arXiv Detail & Related papers (2024-06-13T17:54:38Z) - 3D-HGS: 3D Half-Gaussian Splatting [5.766096863155448]
Photo-realistic 3D Reconstruction is a fundamental problem in 3D computer vision.
We propose to employ 3D Half-Gaussian (3D-HGS) kernels, which can be used as a plug-and-play kernel.
arXiv Detail & Related papers (2024-06-04T19:04:29Z) - Gaussian Splatting Decoder for 3D-aware Generative Adversarial Networks [10.207899254360374]
NeRF-based 3D-aware Generative Adversarial Networks (GANs) have shown very high rendering quality under large representational variety.
rendering with Neural Radiance Fields poses challenges for 3D applications.
We present a novel approach that combines the high rendering quality of NeRF-based 3D-aware GANs with the flexibility and computational advantages of 3DGS.
arXiv Detail & Related papers (2024-04-16T14:48:40Z) - Recent Advances in 3D Gaussian Splatting [31.3820273122585]
3D Gaussian Splatting has greatly accelerated rendering speed of novel view synthesis.
The explicit representation of 3D Gaussian Splatting facilitates editing tasks like dynamic reconstruction, geometry editing, and physical simulation.
We present a literature review of recent 3D Gaussian Splatting methods, which can be roughly classified into 3D reconstruction, 3D editing, and other downstream applications.
arXiv Detail & Related papers (2024-03-17T07:57:08Z) - Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian
Splatting [57.80942520483354]
3D-GS frequently encounters difficulties in accurately modeling specular and anisotropic components.
We introduce Spec-Gaussian, an approach that utilizes an anisotropic spherical Gaussian appearance field instead of spherical harmonics.
Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality.
arXiv Detail & Related papers (2024-02-24T17:22:15Z) - A Survey on 3D Gaussian Splatting [51.96747208581275]
3D Gaussian splatting (GS) has emerged as a transformative technique in the realm of explicit radiance field and computer graphics.
We provide the first systematic overview of the recent developments and critical contributions in the domain of 3D GS.
By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond.
arXiv Detail & Related papers (2024-01-08T13:42:59Z) - Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and
Editing [53.05069432989608]
We present a novel framework for generating 3D human heads with remarkable flexibility.
Our method facilitates the creation of diverse and realistic 3D human heads with fine-grained editing over facial features and expressions.
arXiv Detail & Related papers (2023-12-05T19:05:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.