FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding

• URL: http://arxiv.org/abs/2312.02214v2
• Date: Fri, 29 Mar 2024 16:31:44 GMT
• Title: FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding
• Authors: Jun Xiang, Xuan Gao, Yudong Guo, Juyong Zhang,
• Abstract summary: FlashAvatar is a novel and lightweight 3D animatable avatar representation. It can reconstruct a digital avatar from a short monocular video sequence in minutes. It can render high-fidelity photo-realistic images at 300FPS on a consumer-grade GPU.
• Score: 33.03011612052884
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose FlashAvatar, a novel and lightweight 3D animatable avatar representation that could reconstruct a digital avatar from a short monocular video sequence in minutes and render high-fidelity photo-realistic images at 300FPS on a consumer-grade GPU. To achieve this, we maintain a uniform 3D Gaussian field embedded in the surface of a parametric face model and learn extra spatial offset to model non-surface regions and subtle facial details. While full use of geometric priors can capture high-frequency facial details and preserve exaggerated expressions, proper initialization can help reduce the number of Gaussians, thus enabling super-fast rendering speed. Extensive experimental results demonstrate that FlashAvatar outperforms existing works regarding visual quality and personalized details and is almost an order of magnitude faster in rendering speed. Project page: https://ustc3dv.github.io/FlashAvatar/

Related papers

• DNF-Avatar: Distilling Neural Fields for Real-time Animatable Avatar Relighting [12.917419616798815]
  Creating relightable and an computationable human avatars from monocular videos is a rising research topic with a range of applications. Previous works utilize neural fields together with physically based rendering (PBR), to estimate geometry and disentangle appearance properties of human avatars. To tackle this problem, we proposed to distill the knowledge from implicit neural fields to explicit 2D Gaussian splatting representation.
  arXiv  Detail & Related papers  (2025-04-14T17:59:58Z)
• GraphAvatar: Compact Head Avatars with GNN-Generated 3D Gaussians [14.420981606586237]
  We introduce a method called GraphAvatar that utilizes Graph Neural Networks (GNN) to generate 3D Gaussians for the head avatar. Our method can store the GNN models instead of the 3D Gaussians, significantly reducing the storage overhead to just 10MB. We conduct comprehensive experiments to demonstrate the advantages of GraphAvatar, surpassing existing methods in visual fidelity and storage consumption.
  arXiv  Detail & Related papers  (2024-12-18T16:05:40Z)
• Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives [60.217580865237835]
  3D Gaussian Splatting (3D-GS) is a recent 3D scene reconstruction technique that enables real-time rendering of novel views by modeling scenes as parametric point clouds of differentiable 3D Gaussians. We identify and address two key inefficiencies in 3D-GS to substantially improve rendering speed. Our Speedy-Splat approach combines these techniques to accelerate average rendering speed by a drastic $mathit6.71times$ across scenes from the Mip-NeRF 360, Tanks & Temples, and Deep Blending datasets.
  arXiv  Detail & Related papers  (2024-11-30T20:25:56Z)
• Generalizable and Animatable Gaussian Head Avatar [50.34788590904843]
  We propose Generalizable and Animatable Gaussian head Avatar (GAGAvatar) for one-shot animatable head avatar reconstruction. We generate the parameters of 3D Gaussians from a single image in a single forward pass. Our method exhibits superior performance compared to previous methods in terms of reconstruction quality and expression accuracy.
  arXiv  Detail & Related papers  (2024-10-10T14:29:00Z)
• LightAvatar: Efficient Head Avatar as Dynamic Neural Light Field [58.93692943064746]
  We introduce LightAvatar, the first head avatar model based on neural light fields (NeLFs) LightAvatar renders an image from 3DMM parameters and a camera pose via a single network forward pass, without using mesh or volume rendering.
  arXiv  Detail & Related papers  (2024-09-26T17:00:02Z)
• Interactive Rendering of Relightable and Animatable Gaussian Avatars [37.73483372890271]
  We propose a simple and efficient method to decouple body materials and lighting from multi-view or monocular avatar videos. Our method can render higher quality results at a faster speed on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2024-07-15T13:25:07Z)
• PSAvatar: A Point-based Shape Model for Real-Time Head Avatar Animation with 3D Gaussian Splatting [17.78639236586134]
  PSAvatar is a novel framework for animatable head avatar creation. It employs 3D Gaussian for fine detail representation and high fidelity rendering. We show that PSAvatar can reconstruct high-fidelity head avatars of a variety of subjects and the avatars can be animated in real-time.
  arXiv  Detail & Related papers  (2024-01-23T16:40:47Z)
• GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning [60.33970027554299]
  Gaussian splatting has emerged as a powerful 3D representation that harnesses the advantages of both explicit (mesh) and implicit (NeRF) 3D representations. In this paper, we seek to leverage Gaussian splatting to generate realistic animatable avatars from textual descriptions. Our proposed method, GAvatar, enables the large-scale generation of diverse animatable avatars using only text prompts.
  arXiv  Detail & Related papers  (2023-12-18T18:59:12Z)
• 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting [32.63571465495127]
  We introduce an approach that creates animatable human avatars from monocular videos using 3D Gaussian Splatting (3DGS) We learn a non-rigid network to reconstruct animatable clothed human avatars that can be trained within 30 minutes and rendered at real-time frame rates (50+ FPS) Experimental results show that our method achieves comparable and even better performance compared to state-of-the-art approaches on animatable avatar creation from a monocular input.
  arXiv  Detail & Related papers  (2023-12-14T18:54:32Z)
• ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering [62.81677824868519]
  We propose an animatable Gaussian splatting approach for photorealistic rendering of dynamic humans in real-time. We parameterize the clothed human as animatable 3D Gaussians, which can be efficiently splatted into image space to generate the final rendering. We benchmark ASH with competing methods on pose-controllable avatars, demonstrating that our method outperforms existing real-time methods by a large margin and shows comparable or even better results than offline methods.
  arXiv  Detail & Related papers  (2023-12-10T17:07:37Z)
• FLARE: Fast Learning of Animatable and Relightable Mesh Avatars [64.48254296523977]
  Our goal is to efficiently learn personalized animatable 3D head avatars from videos that are geometrically accurate, realistic, relightable, and compatible with current rendering systems. We introduce FLARE, a technique that enables the creation of animatable and relightable avatars from a single monocular video.
  arXiv  Detail & Related papers  (2023-10-26T16:13:00Z)
• Efficient Meshy Neural Fields for Animatable Human Avatars [87.68529918184494]
  Efficiently digitizing high-fidelity animatable human avatars from videos is a challenging and active research topic. Recent rendering-based neural representations open a new way for human digitization with their friendly usability and photo-varying reconstruction quality. We present EMA, a method that Efficiently learns Meshy neural fields to reconstruct animatable human Avatars.
  arXiv  Detail & Related papers  (2023-03-23T00:15:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.