TAVA: Template-free Animatable Volumetric Actors

• URL: http://arxiv.org/abs/2206.08929v2
• Date: Tue, 21 Jun 2022 03:14:02 GMT
• Title: TAVA: Template-free Animatable Volumetric Actors
• Authors: Ruilong Li, Julian Tanke, Minh Vo, Michael Zollhofer, Jurgen Gall, Angjoo Kanazawa, Christoph Lassner
• Abstract summary: We propose TAVA, a method to create T emplate-free Animatable Volumetric Actors, based on neural representations. Since TAVA does not require a body template, it is applicable to humans as well as other creatures such as animals.
• Score: 29.93065805208324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Coordinate-based volumetric representations have the potential to generate photo-realistic virtual avatars from images. However, virtual avatars also need to be controllable even to a novel pose that may not have been observed. Traditional techniques, such as LBS, provide such a function; yet it usually requires a hand-designed body template, 3D scan data, and limited appearance models. On the other hand, neural representation has been shown to be powerful in representing visual details, but are under explored on deforming dynamic articulated actors. In this paper, we propose TAVA, a method to create T emplate-free Animatable Volumetric Actors, based on neural representations. We rely solely on multi-view data and a tracked skeleton to create a volumetric model of an actor, which can be animated at the test time given novel pose. Since TAVA does not require a body template, it is applicable to humans as well as other creatures such as animals. Furthermore, TAVA is designed such that it can recover accurate dense correspondences, making it amenable to content-creation and editing tasks. Through extensive experiments, we demonstrate that the proposed method generalizes well to novel poses as well as unseen views and showcase basic editing capabilities.

Related papers

• GenCA: A Text-conditioned Generative Model for Realistic and Drivable Codec Avatars [44.8290935585746]
  Photo-realistic and controllable 3D avatars are crucial for various applications such as virtual and mixed reality (VR/MR), telepresence, gaming, and film production. Traditional methods for avatar creation often involve time-consuming scanning and reconstruction processes for each avatar. We propose a text-conditioned generative model that can generate photo-realistic facial avatars of diverse identities.
  arXiv  Detail & Related papers  (2024-08-24T21:25:22Z)
• Deformable 3D Gaussian Splatting for Animatable Human Avatars [50.61374254699761]
  We propose a fully explicit approach to construct a digital avatar from as little as a single monocular sequence. ParDy-Human constitutes an explicit model for realistic dynamic human avatars which requires significantly fewer training views and images. Our avatars learning is free of additional annotations such as Splat masks and can be trained with variable backgrounds while inferring full-resolution images efficiently even on consumer hardware.
  arXiv  Detail & Related papers  (2023-12-22T20:56:46Z)
• Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization [91.52882218901627]
  We propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing. Our method improves upon photo-realism, geometry, and expression accuracy compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-04T17:58:40Z)
• Text-Conditional Contextualized Avatars For Zero-Shot Personalization [47.85747039373798]
  We propose a pipeline that enables personalization of image generation with avatars capturing a user's identity in a delightful way. Our pipeline is zero-shot, avatar texture and style agnostic, and does not require training on the avatar at all. We show, for the first time, how to leverage large-scale image datasets to learn human 3D pose parameters.
  arXiv  Detail & Related papers  (2023-04-14T22:00:44Z)
• AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control [38.959851274747145]
  AvatarCraft is a method for creating a 3D human avatar with a specific identity and artistic style that can be easily animated. We use diffusion models to guide the learning of geometry and texture for a neural avatar based on a single text prompt. We make the human avatar animatable by deforming the neural implicit field with an explicit warping field.
  arXiv  Detail & Related papers  (2023-03-30T17:59:59Z)
• Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors [98.24047528960406]
  We propose a new method for learning a generalized animatable neural representation from a sparse set of multi-view imagery of multiple persons. The learned representation can be used to synthesize novel view images of an arbitrary person from a sparse set of cameras, and further animate them with the user's pose control.
  arXiv  Detail & Related papers  (2022-08-25T07:36:46Z)
• LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies [78.17425779503047]
  We propose a novel neural implicit representation for the human body. It is fully differentiable and optimizable with disentangled shape and pose latent spaces. Our model can be trained and fine-tuned directly on non-watertight raw data with well-designed losses.
  arXiv  Detail & Related papers  (2021-11-30T04:10:57Z)
• MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images [60.56518548286836]
  To generate realistic cloth deformations from novel input poses, watertight meshes or dense full-body scans are usually needed as inputs. We propose an approach that can quickly generate realistic clothed human avatars, represented as controllable neural SDFs, given only monocular depth images.
  arXiv  Detail & Related papers  (2021-06-22T17:30:12Z)
• Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control [80.79820002330457]
  We propose a new method for high-quality synthesis of humans from arbitrary viewpoints and under arbitrary controllable poses. Our method achieves better quality than the state-of-the-arts on playback as well as novel pose synthesis, and can even generalize well to new poses that starkly differ from the training poses.
  arXiv  Detail & Related papers  (2021-06-03T17:40:48Z)
• Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction [9.747648609960185]
  We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoints or head-poses is required.
  arXiv  Detail & Related papers  (2020-12-05T16:01:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.