Instant Volumetric Head Avatars

• URL: http://arxiv.org/abs/2211.12499v2
• Date: Thu, 23 Mar 2023 13:16:06 GMT
• Title: Instant Volumetric Head Avatars
• Authors: Wojciech Zielonka, Timo Bolkart, Justus Thies
• Abstract summary: We present Instant Volumetric Head Avatars (INSTA), a novel approach for reconstructing photo-realistic digital avatars instantaneously. Our pipeline is trained on a single monocular RGB portrait video that observes the subject under different expressions and views. INSTA reconstructs a digital avatar in less than 10 minutes on modern GPU hardware, which is orders of magnitude faster than previous solutions.
• Score: 20.782425305421505
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Instant Volumetric Head Avatars (INSTA), a novel approach for reconstructing photo-realistic digital avatars instantaneously. INSTA models a dynamic neural radiance field based on neural graphics primitives embedded around a parametric face model. Our pipeline is trained on a single monocular RGB portrait video that observes the subject under different expressions and views. While state-of-the-art methods take up to several days to train an avatar, our method can reconstruct a digital avatar in less than 10 minutes on modern GPU hardware, which is orders of magnitude faster than previous solutions. In addition, it allows for the interactive rendering of novel poses and expressions. By leveraging the geometry prior of the underlying parametric face model, we demonstrate that INSTA extrapolates to unseen poses. In quantitative and qualitative studies on various subjects, INSTA outperforms state-of-the-art methods regarding rendering quality and training time.

Related papers

• GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations [54.94362657501809]
  We propose a new method to generate highly dynamic and deformable human head avatars from multi-view imagery in real-time. At the core of our method is a hierarchical representation of head models that allows to capture the complex dynamics of facial expressions and head movements. We train this coarse-to-fine facial avatar model along with the head pose as a learnable parameter in an end-to-end framework.
  arXiv  Detail & Related papers  (2024-09-18T13:05:43Z)
• HQ3DAvatar: High Quality Controllable 3D Head Avatar [65.70885416855782]
  This paper presents a novel approach to building highly photorealistic digital head avatars. Our method learns a canonical space via an implicit function parameterized by a neural network. At test time, our method is driven by a monocular RGB video.
  arXiv  Detail & Related papers  (2023-03-25T13:56:33Z)
• Real-time volumetric rendering of dynamic humans [83.08068677139822]
  We present a method for fast 3D reconstruction and real-time rendering of dynamic humans from monocular videos. Our method can reconstruct a dynamic human in less than 3h using a single GPU, compared to recent state-of-the-art alternatives that take up to 72h. A novel local ray marching rendering allows visualizing the neural human on a mobile VR device at 40 frames per second with minimal loss of visual quality.
  arXiv  Detail & Related papers  (2023-03-21T14:41:25Z)
• DINAR: Diffusion Inpainting of Neural Textures for One-Shot Human Avatars [7.777410338143783]
  We present an approach for creating realistic rigged fullbody avatars from single RGB images. Our method uses neural textures combined with the SMPL-X body model to achieve photo-realistic quality of avatars. In the experiments, our approach achieves state-of-the-art rendering quality and good generalization to new poses and viewpoints.
  arXiv  Detail & Related papers  (2023-03-16T15:04:10Z)
• MegaPortraits: One-shot Megapixel Neural Head Avatars [7.05068904295608]
  We propose a set of new neural architectures and training methods that can leverage both medium-resolution video data and high-resolution image data. We show how a trained high-resolution neural avatar model can be distilled into a lightweight student model which runs in real-time. Real-time operation and identity lock are essential for many practical applications head avatar systems.
  arXiv  Detail & Related papers  (2022-07-15T17:32:37Z)
• I M Avatar: Implicit Morphable Head Avatars from Videos [68.13409777995392]
  We propose IMavatar, a novel method for learning implicit head avatars from monocular videos. Inspired by the fine-grained control mechanisms afforded by conventional 3DMMs, we represent the expression- and pose-related deformations via learned blendshapes and skinning fields. We show quantitatively and qualitatively that our method improves geometry and covers a more complete expression space compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-12-14T15:30:32Z)
• Neural Head Avatars from Monocular RGB Videos [0.0]
  We present a novel neural representation that explicitly models the surface geometry and appearance of an animatable human avatar. Our representation can be learned from a monocular RGB portrait video that features a range of different expressions and views.
  arXiv  Detail & Related papers  (2021-12-02T19:01:05Z)
• MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images [60.56518548286836]
  To generate realistic cloth deformations from novel input poses, watertight meshes or dense full-body scans are usually needed as inputs. We propose an approach that can quickly generate realistic clothed human avatars, represented as controllable neural SDFs, given only monocular depth images.
  arXiv  Detail & Related papers  (2021-06-22T17:30:12Z)
• Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control [80.79820002330457]
  We propose a new method for high-quality synthesis of humans from arbitrary viewpoints and under arbitrary controllable poses. Our method achieves better quality than the state-of-the-arts on playback as well as novel pose synthesis, and can even generalize well to new poses that starkly differ from the training poses.
  arXiv  Detail & Related papers  (2021-06-03T17:40:48Z)
• Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction [9.747648609960185]
  We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoints or head-poses is required.
  arXiv  Detail & Related papers  (2020-12-05T16:01:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.