GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians

• URL: http://arxiv.org/abs/2312.02134v3
• Date: Sat, 30 Mar 2024 04:22:34 GMT
• Title: GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians
• Authors: Liangxiao Hu, Hongwen Zhang, Yuxiang Zhang, Boyao Zhou, Boning Liu, Shengping Zhang, Liqiang Nie,
• Abstract summary: We present an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video. GustafAvatar is validated on both the public dataset and our collected dataset.
• Score: 51.46168990249278
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present GaussianAvatar, an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video. We start by introducing animatable 3D Gaussians to explicitly represent humans in various poses and clothing styles. Such an explicit and animatable representation can fuse 3D appearances more efficiently and consistently from 2D observations. Our representation is further augmented with dynamic properties to support pose-dependent appearance modeling, where a dynamic appearance network along with an optimizable feature tensor is designed to learn the motion-to-appearance mapping. Moreover, by leveraging the differentiable motion condition, our method enables a joint optimization of motions and appearances during avatar modeling, which helps to tackle the long-standing issue of inaccurate motion estimation in monocular settings. The efficacy of GaussianAvatar is validated on both the public dataset and our collected dataset, demonstrating its superior performances in terms of appearance quality and rendering efficiency.

Related papers

• iHuman: Instant Animatable Digital Humans From Monocular Videos [16.98924995658091]
  We present a fast, simple, yet effective method for creating animatable 3D digital humans from monocular videos. This work achieves and illustrates the need of accurate 3D mesh-type modelling of the human body. Our method is faster by an order of magnitude (in terms of training time) than its closest competitor.
  arXiv  Detail & Related papers  (2024-07-15T18:51:51Z)
• PICA: Physics-Integrated Clothed Avatar [30.277983921620663]
  We introduce PICA, a novel representation for high-fidelity animatable clothed human avatars with physics-accurate dynamics, even for loose clothing. Our method achieves high-fidelity rendering of human bodies in complex and novel driving poses, significantly outperforming previous methods under the same settings.
  arXiv  Detail & Related papers  (2024-07-07T10:23:21Z)
• NPGA: Neural Parametric Gaussian Avatars [46.52887358194364]
  We propose a data-driven approach to create high-fidelity controllable avatars from multi-view video recordings. We build our method around 3D Gaussian splatting for its highly efficient rendering and to inherit the topological flexibility of point clouds. We evaluate our method on the public NeRSemble dataset, demonstrating that NPGA significantly outperforms the previous state-of-the-art avatars on the self-reenactment task by 2.6 PSNR.
  arXiv  Detail & Related papers  (2024-05-29T17:58:09Z)
• GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh [97.47701169876272]
  GoMAvatar is a novel approach for real-time, memory-efficient, high-quality human modeling. GoMAvatar matches or surpasses current monocular human modeling algorithms in rendering quality.
  arXiv  Detail & Related papers  (2024-04-11T17:59:57Z)
• Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance [25.346255905155424]
  We introduce a methodology for human image animation by leveraging a 3D human parametric model within a latent diffusion framework. By representing the 3D human parametric model as the motion guidance, we can perform parametric shape alignment of the human body between the reference image and the source video motion. Our approach also exhibits superior generalization capabilities on the proposed in-the-wild dataset.
  arXiv  Detail & Related papers  (2024-03-21T18:52:58Z)
• Deformable 3D Gaussian Splatting for Animatable Human Avatars [50.61374254699761]
  We propose a fully explicit approach to construct a digital avatar from as little as a single monocular sequence. ParDy-Human constitutes an explicit model for realistic dynamic human avatars which requires significantly fewer training views and images. Our avatars learning is free of additional annotations such as Splat masks and can be trained with variable backgrounds while inferring full-resolution images efficiently even on consumer hardware.
  arXiv  Detail & Related papers  (2023-12-22T20:56:46Z)
• Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing [53.05069432989608]
  We present a novel framework for generating 3D human heads with remarkable flexibility. Our method facilitates the creation of diverse and realistic 3D human heads with fine-grained editing over facial features and expressions.
  arXiv  Detail & Related papers  (2023-12-05T19:05:58Z)
• Real-time Deep Dynamic Characters [95.5592405831368]
  We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing. We show that our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches.
  arXiv  Detail & Related papers  (2021-05-04T23:28:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.