PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence

• URL: http://arxiv.org/abs/2203.01754v1
• Date: Thu, 3 Mar 2022 15:04:55 GMT
• Title: PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence
• Authors: Zijian Dong, Chen Guo, Jie Song, Xu Chen, Andreas Geiger, Otmar Hilliges
• Abstract summary: We present a novel method to learn Personalized Implicit Neural Avatars (PINA) from a short RGB-D sequence. PINA does not require complete scans, nor does it require a prior learned from large datasets of clothed humans. We propose a method to learn the shape and non-rigid deformations via a pose-conditioned implicit surface and a deformation field.
• Score: 60.46092534331516
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel method to learn Personalized Implicit Neural Avatars (PINA) from a short RGB-D sequence. This allows non-expert users to create a detailed and personalized virtual copy of themselves, which can be animated with realistic clothing deformations. PINA does not require complete scans, nor does it require a prior learned from large datasets of clothed humans. Learning a complete avatar in this setting is challenging, since only few depth observations are available, which are noisy and incomplete (i.e.only partial visibility of the body per frame). We propose a method to learn the shape and non-rigid deformations via a pose-conditioned implicit surface and a deformation field, defined in canonical space. This allows us to fuse all partial observations into a single consistent canonical representation. Fusion is formulated as a global optimization problem over the pose, shape and skinning parameters. The method can learn neural avatars from real noisy RGB-D sequences for a diverse set of people and clothing styles and these avatars can be animated given unseen motion sequences.

Related papers

• NECA: Neural Customizable Human Avatar [36.69012172745299]
  We introduce NECA, an approach capable of learning versatile human representation from monocular or sparse-view videos. The core of our approach is to represent humans in complementary dual spaces and predict disentangled neural fields of geometry, albedo, shadow, as well as an external lighting.
  arXiv  Detail & Related papers  (2024-03-15T14:23:06Z)
• Deformable 3D Gaussian Splatting for Animatable Human Avatars [50.61374254699761]
  We propose a fully explicit approach to construct a digital avatar from as little as a single monocular sequence. ParDy-Human constitutes an explicit model for realistic dynamic human avatars which requires significantly fewer training views and images. Our avatars learning is free of additional annotations such as Splat masks and can be trained with variable backgrounds while inferring full-resolution images efficiently even on consumer hardware.
  arXiv  Detail & Related papers  (2023-12-22T20:56:46Z)
• AniPixel: Towards Animatable Pixel-Aligned Human Avatar [65.7175527782209]
  AniPixel is a novel animatable and generalizable human avatar reconstruction method. We propose a neural skinning field based on skeleton-driven deformation to establish the target-to-canonical and canonical-to-observation correspondences. Experiments show that AniPixel renders comparable novel views while delivering better novel pose animation results than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-07T11:04:14Z)
• ARAH: Animatable Volume Rendering of Articulated Human SDFs [37.48271522183636]
  We propose a model to create animatable clothed human avatars with detailed geometry that generalize well to out-of-distribution poses. Our algorithm enables efficient point sampling and accurate point canonicalization while generalizing well to unseen poses. Our method achieves state-of-the-art performance on geometry and appearance reconstruction while creating animatable avatars.
  arXiv  Detail & Related papers  (2022-10-18T17:56:59Z)
• I M Avatar: Implicit Morphable Head Avatars from Videos [68.13409777995392]
  We propose IMavatar, a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose-related deformations via learned blendshapes and skinning fields. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-14T15:30:32Z)
• Animatable Neural Radiance Fields from Monocular RGB Video [72.6101766407013]
  We present animatable neural radiance fields for detailed human avatar creation from monocular videos. Our approach extends neural radiance fields to the dynamic scenes with human movements via introducing explicit pose-guided deformation. In experiments we show that the proposed approach achieves 1) implicit human geometry and appearance reconstruction with high-quality details, 2) photo-realistic rendering of the human from arbitrary views, and 3) animation of the human with arbitrary poses.
  arXiv  Detail & Related papers  (2021-06-25T13:32:23Z)
• SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks [54.94737477860082]
  We present an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar. SCANimate does not rely on a customized mesh template or surface mesh registration. Our method can be applied to pose-aware appearance modeling to generate a fully textured avatar.
  arXiv  Detail & Related papers  (2021-04-07T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.