PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging
- URL: http://arxiv.org/abs/2106.10893v1
- Date: Mon, 21 Jun 2021 07:21:20 GMT
- Title: PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging
- Authors: Yuwei Li, Minye Wu, Yuyao Zhang, Lan Xu, Jingyi Yu
- Abstract summary: We present PIANO, the first parametric bone model of human hands from MRI data.
Our PIANO model is biologically anatomically correct, simple to animate, and differentiable, achieving more precise modeling of the inner hand kinematic structure.
- Score: 43.66613296379493
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Hand modeling is critical for immersive VR/AR, action understanding, or human
healthcare. Existing parametric models account only for hand shape, pose, or
texture, without modeling the anatomical attributes like bone, which is
essential for realistic hand biomechanics analysis. In this paper, we present
PIANO, the first parametric bone model of human hands from MRI data. Our PIANO
model is biologically correct, simple to animate, and differentiable, achieving
more anatomically precise modeling of the inner hand kinematic structure in a
data-driven manner than the traditional hand models based on the outer surface
only. Furthermore, our PIANO model can be applied in neural network layers to
enable training with a fine-grained semantic loss, which opens up the new task
of data-driven fine-grained hand bone anatomic and semantic understanding from
MRI or even RGB images. We make our model publicly available.
Related papers
- MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints [50.61346764110482]
We integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create MS-MANO.
This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories.
We also propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron network.
arXiv Detail & Related papers (2024-04-16T02:18:18Z) - 3D Kinematics Estimation from Video with a Biomechanical Model and
Synthetic Training Data [4.130944152992895]
We propose a novel biomechanics-aware network that directly outputs 3D kinematics from two input views.
Our experiments demonstrate that the proposed approach, only trained on synthetic data, outperforms previous state-of-the-art methods.
arXiv Detail & Related papers (2024-02-20T17:33:40Z) - Anatomically Constrained Implicit Face Models [7.141905869633729]
We present a novel use case for such implicit representations in the context of learning anatomically constrained face models.
We propose the anatomical implicit face model; an ensemble of networks that jointly learn to model the facial anatomy and the skin surface with high-fidelity.
We demonstrate the usefulness of our approach in several tasks ranging from shape fitting, shape editing, and performance.
arXiv Detail & Related papers (2023-12-12T18:59:21Z) - Direct and inverse modeling of soft robots by learning a condensed FEM
model [3.4696964555947694]
We propose a learning-based approach to obtain a compact but sufficiently rich mechanical representation.
We show how to couple some models learned individually in particular on an example of a gripper composed of two soft fingers.
This work opens new perspectives, namely for the embedded control of soft robots, but also for their design.
arXiv Detail & Related papers (2023-07-21T08:07:16Z) - HandNeRF: Neural Radiance Fields for Animatable Interacting Hands [122.32855646927013]
We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands.
We conduct extensive experiments to verify the merits of our proposed HandNeRF and report a series of state-of-the-art results.
arXiv Detail & Related papers (2023-03-24T06:19:19Z) - NIMBLE: A Non-rigid Hand Model with Bones and Muscles [41.19718491215149]
We present NIMBLE, a novel parametric hand model that includes the missing key components.
NIMBLE consists of 20 bones as triangular meshes, 7 muscle groups as tetrahedral meshes, and a skin mesh.
We demonstrate applying NIMBLE to modeling, rendering, and visual inference tasks.
arXiv Detail & Related papers (2022-02-09T15:57:21Z) - A Skeleton-Driven Neural Occupancy Representation for Articulated Hands [49.956892429789775]
Hand ArticuLated Occupancy (HALO) is a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces.
We demonstrate the applicability of HALO to the task of conditional generation of hands that grasp 3D objects.
arXiv Detail & Related papers (2021-09-23T14:35:19Z) - Learning Local Recurrent Models for Human Mesh Recovery [50.85467243778406]
We present a new method for video mesh recovery that divides the human mesh into several local parts following the standard skeletal model.
We then model the dynamics of each local part with separate recurrent models, with each model conditioned appropriately based on the known kinematic structure of the human body.
This results in a structure-informed local recurrent learning architecture that can be trained in an end-to-end fashion with available annotations.
arXiv Detail & Related papers (2021-07-27T14:30:33Z) - Combining Implicit Function Learning and Parametric Models for 3D Human
Reconstruction [123.62341095156611]
Implicit functions represented as deep learning approximations are powerful for reconstructing 3D surfaces.
Such features are essential in building flexible models for both computer graphics and computer vision.
We present methodology that combines detail-rich implicit functions and parametric representations.
arXiv Detail & Related papers (2020-07-22T13:46:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.