GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation

• URL: http://arxiv.org/abs/2312.01632v4
• Date: Thu, 30 May 2024 02:19:25 GMT
• Title: GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation
• Authors: Jie Wang, Jiu-Cheng Xie, Xianyan Li, Feng Xu, Chi-Man Pun, Hao Gao,
• Abstract summary: This paper presents a framework to model the actional human head with anisotropic 3D Gaussians. In experiments, our method can produce high-fidelity renderings, outperforming state-of-the-art approaches in reconstruction, cross-identity reenactment, and novel view synthesis tasks.
• Score: 35.39887092268696
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Constructing vivid 3D head avatars for given subjects and realizing a series of animations on them is valuable yet challenging. This paper presents GaussianHead, which models the actional human head with anisotropic 3D Gaussians. In our framework, a motion deformation field and multi-resolution tri-plane are constructed respectively to deal with the head's dynamic geometry and complex texture. Notably, we impose an exclusive derivation scheme on each Gaussian, which generates its multiple doppelgangers through a set of learnable parameters for position transformation. With this design, we can compactly and accurately encode the appearance information of Gaussians, even those fitting the head's particular components with sophisticated structures. In addition, an inherited derivation strategy for newly added Gaussians is adopted to facilitate training acceleration. Extensive experiments show that our method can produce high-fidelity renderings, outperforming state-of-the-art approaches in reconstruction, cross-identity reenactment, and novel view synthesis tasks. Our code is available at: https://github.com/chiehwangs/gaussian-head.

Related papers

• Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting [33.01987451251659]
  3D Gaussian Splatting (3DGS) has emerged as a promising technique capable of real-time rendering with high-quality 3D reconstruction. Despite its potential, 3DGS encounters challenges, including needle-like artifacts, suboptimal geometries, and inaccurate normals. We introduce effective rank as a regularization, which constrains the structure of the Gaussians.
  arXiv  Detail & Related papers  (2024-06-17T15:51:59Z)
• $E^{3}$Gen: Efficient, Expressive and Editable Avatars Generation [71.72171053129655]
  This paper aims to introduce 3D Gaussian for efficient, expressive, and editable digital avatar generation. We propose a novel avatar generation method named $E3$Gen to effectively address these challenges. Our method achieves superior performance in avatar generation and enables expressive full-body pose control and editing.
  arXiv  Detail & Related papers  (2024-05-29T15:43:49Z)
• Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping [12.308192525760667]
  Existing methods only reconstruct a head without the body, substantially limiting their application scenarios. We propose to model the body part with a neural texture that consists of coarse and pose-dependent fine colors. We demonstrate that Gaussian Head & Shoulders can fit the high-frequency details on the clothed upper body with high fidelity and potentially improve the accuracy and fidelity of the head region.
  arXiv  Detail & Related papers  (2024-05-20T14:39:49Z)
• Gaussian Opacity Fields: Efficient and Compact Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and compact surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting [57.80942520483354]
  3D-GS frequently encounters difficulties in accurately modeling specular and anisotropic components. We introduce Spec-Gaussian, an approach that utilizes an anisotropic spherical Gaussian appearance field instead of spherical harmonics. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality.
  arXiv  Detail & Related papers  (2024-02-24T17:22:15Z)
• Mesh-based Gaussian Splatting for Real-time Large-scale Deformation [58.18290393082119]
  It is challenging for users to directly deform or manipulate implicit representations with large deformations in the real-time fashion. We develop a novel GS-based method that enables interactive deformation. Our approach achieves high-quality reconstruction and effective deformation, while maintaining the promising rendering results at a high frame rate.
  arXiv  Detail & Related papers  (2024-02-07T12:36:54Z)
• GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning [60.33970027554299]
  Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts.
  arXiv  Detail & Related papers  (2023-12-18T18:59:12Z)
• MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar [44.125711148560605]
  MonoGaussianAvatar is a novel approach that harnesses 3D Gaussian point representation and a Gaussian deformation field to learn explicit head avatars from monocular portrait videos. Experiments demonstrate the superior performance of our method, which achieves state-of-the-art results among previous methods.
  arXiv  Detail & Related papers  (2023-12-07T18:59:31Z)
• Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing [53.05069432989608]
  We present a novel framework for generating 3D human heads with remarkable flexibility. Our method facilitates the creation of diverse and realistic 3D human heads with fine-grained editing over facial features and expressions.
  arXiv  Detail & Related papers  (2023-12-05T19:05:58Z)
• Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians [41.86540576028268]
  We propose controllable 3D Gaussian Head Avatars for lightweight sparse-view setups. We show our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution even under exaggerated expressions.
  arXiv  Detail & Related papers  (2023-12-05T11:01:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.