AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capture

• URL: http://arxiv.org/abs/2207.02031v1
• Date: Tue, 5 Jul 2022 13:21:01 GMT
• Title: AvatarCap: Animatable Avatar Conditioned Monocular Human Volumetric Capture
• Authors: Zhe Li, Zerong Zheng, Hongwen Zhang, Chaonan Ji, Yebin Liu
• Abstract summary: AvatarCap is a novel framework that introduces animatable avatars into the capture pipeline for high-fidelity reconstruction in both visible and invisible regions. Our method integrates information from both the image observation and the avatar prior, and accordingly recon-structs high-fidelity 3D textured models with dynamic details regardless of the visibility.
• Score: 36.10436374741757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To address the ill-posed problem caused by partial observations in monocular human volumetric capture, we present AvatarCap, a novel framework that introduces animatable avatars into the capture pipeline for high-fidelity reconstruction in both visible and invisible regions. Our method firstly creates an animatable avatar for the subject from a small number (~20) of 3D scans as a prior. Then given a monocular RGB video of this subject, our method integrates information from both the image observation and the avatar prior, and accordingly recon-structs high-fidelity 3D textured models with dynamic details regardless of the visibility. To learn an effective avatar for volumetric capture from only few samples, we propose GeoTexAvatar, which leverages both geometry and texture supervisions to constrain the pose-dependent dynamics in a decomposed implicit manner. An avatar-conditioned volumetric capture method that involves a canonical normal fusion and a reconstruction network is further proposed to integrate both image observations and avatar dynamics for high-fidelity reconstruction in both observed and invisible regions. Overall, our method enables monocular human volumetric capture with detailed and pose-dependent dynamics, and the experiments show that our method outperforms state of the art. Code is available at https://github.com/lizhe00/AvatarCap.

Related papers

• NPGA: Neural Parametric Gaussian Avatars [46.52887358194364]
  We propose a data-driven approach to create high-fidelity controllable avatars from multi-view video recordings. We build our method around 3D Gaussian splatting for its highly efficient rendering and to inherit the topological flexibility of point clouds. We evaluate our method on the public NeRSemble dataset, demonstrating that NPGA significantly outperforms the previous state-of-the-art avatars on the self-reenactment task by 2.6 PSNR.
  arXiv  Detail & Related papers  (2024-05-29T17:58:09Z)
• HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos [52.23323966700072]
  We present a framework for acquiring human avatars that are attached with high-resolution physically-based material textures and mesh from monocular video. Our method introduces a novel information fusion strategy to combine the information from the monocular video and synthesize virtual multi-view images. Experiments show that our approach outperforms previous representations in terms of high fidelity, and this explicit result supports deployment on common triangulars.
  arXiv  Detail & Related papers  (2024-05-18T11:49:09Z)
• GPAvatar: Generalizable and Precise Head Avatar from Image(s) [71.555405205039]
  GPAvatar is a framework that reconstructs 3D head avatars from one or several images in a single forward pass. The proposed method achieves faithful identity reconstruction, precise expression control, and multi-view consistency.
  arXiv  Detail & Related papers  (2024-01-18T18:56:34Z)
• Deformable 3D Gaussian Splatting for Animatable Human Avatars [50.61374254699761]
  We propose a fully explicit approach to construct a digital avatar from as little as a single monocular sequence. ParDy-Human constitutes an explicit model for realistic dynamic human avatars which requires significantly fewer training views and images. Our avatars learning is free of additional annotations such as Splat masks and can be trained with variable backgrounds while inferring full-resolution images efficiently even on consumer hardware.
  arXiv  Detail & Related papers  (2023-12-22T20:56:46Z)
• MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar [44.125711148560605]
  MonoGaussianAvatar is a novel approach that harnesses 3D Gaussian point representation and a Gaussian deformation field to learn explicit head avatars from monocular portrait videos. Experiments demonstrate the superior performance of our method, which achieves state-of-the-art results among previous methods.
  arXiv  Detail & Related papers  (2023-12-07T18:59:31Z)
• GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians [51.46168990249278]
  We present an efficient approach to creating realistic human avatars with dynamic 3D appearances from a single video. GustafAvatar is validated on both the public dataset and our collected dataset.
  arXiv  Detail & Related papers  (2023-12-04T18:55:45Z)
• HAVE-FUN: Human Avatar Reconstruction from Few-Shot Unconstrained Images [33.298962236215964]
  We study the reconstruction of human avatars from a few-shot unconstrained photo album. For handling dynamic data, we integrate a skinning mechanism with deep marching tetrahedra. Our framework, called HaveFun, can undertake avatar reconstruction, rendering, and animation.
  arXiv  Detail & Related papers  (2023-11-27T10:01:31Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• Drivable Volumetric Avatars using Texel-Aligned Features [52.89305658071045]
  Photo telepresence requires both high-fidelity body modeling and faithful driving to enable dynamically synthesized appearance. We propose an end-to-end framework that addresses two core challenges in modeling and driving full-body avatars of real people.
  arXiv  Detail & Related papers  (2022-07-20T09:28:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.