NIMBLE: A Non-rigid Hand Model with Bones and Muscles

• URL: http://arxiv.org/abs/2202.04533v1
• Date: Wed, 9 Feb 2022 15:57:21 GMT
• Title: NIMBLE: A Non-rigid Hand Model with Bones and Muscles
• Authors: Yuwei Li, Longwen Zhang, Zesong Qiu, Yingwenqi Jiang, Yuyao Zhang, Nianyi Li, Yuexin Ma, Lan Xu, Jingyi Yu
• Abstract summary: We present NIMBLE, a novel parametric hand model that includes the missing key components. NIMBLE consists of 20 bones as triangular meshes, 7 muscle groups as tetrahedral meshes, and a skin mesh. We demonstrate applying NIMBLE to modeling, rendering, and visual inference tasks.
• Score: 41.19718491215149
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Emerging Metaverse applications demand reliable, accurate, and photorealistic reproductions of human hands to perform sophisticated operations as if in the physical world. While real human hand represents one of the most intricate coordination between bones, muscle, tendon, and skin, state-of-the-art techniques unanimously focus on modeling only the skeleton of the hand. In this paper, we present NIMBLE, a novel parametric hand model that includes the missing key components, bringing 3D hand model to a new level of realism. We first annotate muscles, bones and skins on the recent Magnetic Resonance Imaging hand (MRI-Hand) dataset and then register a volumetric template hand onto individual poses and subjects within the dataset. NIMBLE consists of 20 bones as triangular meshes, 7 muscle groups as tetrahedral meshes, and a skin mesh. Via iterative shape registration and parameter learning, it further produces shape blend shapes, pose blend shapes, and a joint regressor. We demonstrate applying NIMBLE to modeling, rendering, and visual inference tasks. By enforcing the inner bones and muscles to match anatomic and kinematic rules, NIMBLE can animate 3D hands to new poses at unprecedented realism. To model the appearance of skin, we further construct a photometric HandStage to acquire high-quality textures and normal maps to model wrinkles and palm print. Finally, NIMBLE also benefits learning-based hand pose and shape estimation by either synthesizing rich data or acting directly as a differentiable layer in the inference network.

Related papers

• Generating Holistic 3D Human Motion from Speech [97.11392166257791]
  We build a high-quality dataset of 3D holistic body meshes with synchronous speech. We then define a novel speech-to-motion generation framework in which the face, body, and hands are modeled separately.
  arXiv  Detail & Related papers  (2022-12-08T17:25:19Z)
• LISA: Learning Implicit Shape and Appearance of Hands [33.477530463242275]
  This paper proposes a do-it-all neural model of human hands, named LISA. The model can capture accurate hand shape and appearance, generalize to arbitrary hand subjects, provide dense surface correspondences, be reconstructed from images in the wild and easily animated.
  arXiv  Detail & Related papers  (2022-04-04T17:59:03Z)
• Embodied Hands: Modeling and Capturing Hands and Bodies Together [61.32931890166915]
  Humans move their hands and bodies together to communicate and solve tasks. Most methods treat the 3D modeling and tracking of bodies and hands separately. We formulate a model of hands and bodies interacting together and fit it to full-body 4D sequences.
  arXiv  Detail & Related papers  (2022-01-07T18:59:32Z)
• A Skeleton-Driven Neural Occupancy Representation for Articulated Hands [49.956892429789775]
  Hand ArticuLated Occupancy (HALO) is a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces. We demonstrate the applicability of HALO to the task of conditional generation of hands that grasp 3D objects.
  arXiv  Detail & Related papers  (2021-09-23T14:35:19Z)
• imGHUM: Implicit Generative Models of 3D Human Shape and Articulated Pose [42.4185273307021]
  We present imGHUM, the first holistic generative model of 3D human shape and articulated pose. We model the full human body implicitly as a function zero-level-set and without the use of an explicit template mesh.
  arXiv  Detail & Related papers  (2021-08-24T17:08:28Z)
• Detailed Avatar Recovery from Single Image [50.82102098057822]
  This paper presents a novel framework to recover emphdetailed avatar from a single image. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation framework. Our method can restore detailed human body shapes with complete textures beyond skinned models.
  arXiv  Detail & Related papers  (2021-08-06T03:51:26Z)
• PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging [43.66613296379493]
  We present PIANO, the first parametric bone model of human hands from MRI data. Our PIANO model is biologically anatomically correct, simple to animate, and differentiable, achieving more precise modeling of the inner hand kinematic structure.
  arXiv  Detail & Related papers  (2021-06-21T07:21:20Z)
• Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control [80.79820002330457]
  We propose a new method for high-quality synthesis of humans from arbitrary viewpoints and under arbitrary controllable poses. Our method achieves better quality than the state-of-the-arts on playback as well as novel pose synthesis, and can even generalize well to new poses that starkly differ from the training poses.
  arXiv  Detail & Related papers  (2021-06-03T17:40:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.