From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans

• URL: http://arxiv.org/abs/2509.06607v1
• Date: Mon, 08 Sep 2025 12:24:27 GMT
• Title: From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans
• Abstract summary: We develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton.<n>We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods.
• Score: 50.014530130312714
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Great progress has been made in estimating 3D human pose and shape from images and video by training neural networks to directly regress the parameters of parametric human models like SMPL. However, existing body models have simplified kinematic structures that do not correspond to the true joint locations and articulations in the human skeletal system, limiting their potential use in biomechanics. On the other hand, methods for estimating biomechanically accurate skeletal motion typically rely on complex motion capture systems and expensive optimization methods. What is needed is a parametric 3D human model with a biomechanically accurate skeletal structure that can be easily posed. To that end, we develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton. To enable this, we need training data of skeletons inside SMPL meshes in diverse poses. We build such a dataset by optimizing biomechanically accurate skeletons inside SMPL meshes from AMASS sequences. We then learn a regressor from SMPL mesh vertices to the optimized joint locations and bone rotations. Finally, we re-parametrize the SMPL mesh with the new kinematic parameters. The resulting SKEL model is animatable like SMPL but with fewer, and biomechanically-realistic, degrees of freedom. We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods. By fitting SKEL to SMPL meshes we are able to "upgrade" existing human pose and shape datasets to include biomechanical parameters. SKEL provides a new tool to enable biomechanics in the wild, while also providing vision and graphics researchers with a better constrained and more realistic model of human articulation. The model, code, and data are available for research at https://skel.is.tue.mpg.de..

Related papers

• Biomechanical 3D Body: Self-Supervised Distillation of Biomechanical Pose from a 3D Body Foundation Model [15.358487825950212]
  We extend an existing foundation model, SAM-3D-Body, with an additional biomechanical prediction head that regresses the joint angles and scales of a biomechanical model.<n>To overcome this, we supervise biomechanical outputs with in-loop optimized targets from a Levenberg-Marquardt solver performing inverse kinematics fits against markers from the mesh predictions.<n>We validated this model on biomechanical fits to two marker-based datasets, MoVi and BioCV, as well as movements from a clinical cohort captured with multiview markerless motion capture.
  arXiv  Detail & Related papers  (2026-08-30T17:46:11Z)
• MotionDreamer: Universal Skeletal Motion Generation for 3D Rigged Shapes [52.45449797329548]
  MotionDreamer is a diffusion-based framework for category-agnostic skeletal animation generation from 2D video guidance.<n>We propose a structural-semantic injection mechanism to bridge the gap between 2D visual motion cues and heterogeneous 3D skeletal structures.<n>Our approach significantly outperforms existing methods, setting a new state-of-the-art benchmark for robust and efficient 4D asset generation.
  arXiv  Detail & Related papers  (2026-06-01T00:42:31Z)
• SKEL-CF: Coarse-to-Fine Biomechanical Skeleton and Surface Mesh Recovery [33.63204394371213]
  We introduce SKEL-CF, a coarse-to-fine framework for SKEL parameter estimation.<n>SKEL-CF employs a transformer-based encoder-decoder architecture, where the encoder predicts coarse camera and SKEL parameters.<n>Our results establish SKEL-CF as a scalable and anatomically faithful framework for human motion analysis.
  arXiv  Detail & Related papers  (2025-11-25T10:33:17Z)
• Reconstructing Humans with a Biomechanically Accurate Skeleton [55.06027148976482]
  We introduce a method for reconstructing 3D humans from a single image using a biomechanically accurate skeleton model.<n>Compared to state-of-the-art methods for 3D human mesh recovery, our model achieves competitive performance on standard benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T17:56:24Z)
• BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos [6.280386490530478]
  BioPose is a learning-based framework for predicting biomechanically accurate 3D human pose directly from monocular videos.<n>It includes a Multi-Query Human Mesh Recovery model (MQ-HMR), a Neural Inverse Kinematics (NeurIK) model, and a 2D-informed pose refinement technique.<n> Experiments on benchmark datasets demonstrate that BioPose significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-14T02:56:19Z)
• ToMiE: Towards Explicit Exoskeleton for the Reconstruction of Complicated 3D Human Avatars [41.23897822168498]
  We propose a growth strategy that enables the joint tree of the skeleton to expand adaptively.<n>Specifically, our method, called ToMiE, consists of parent joints localization and external joints optimization.<n>ToMiE manages to outperform other methods across various cases with hand-held objects and loose-fitting clothing, not only in rendering quality but also by offering free animation of grown joints.
  arXiv  Detail & Related papers  (2024-10-10T16:25:52Z)
• MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints [50.61346764110482]
  We integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We also propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron network.
  arXiv  Detail & Related papers  (2024-04-16T02:18:18Z)
• 3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data [4.130944152992895]
  We propose a novel biomechanics-aware network that directly outputs 3D kinematics from two input views. Our experiments demonstrate that the proposed approach, only trained on synthetic data, outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-20T17:33:40Z)
• OSSO: Obtaining Skeletal Shape from Outside [50.47978215230605]
  OSSO (Obtaining Skeletal Shape from Outside) is the first to learn the mapping from the 3D body surface to the internal skeleton from real data. To these, we fit a parametric 3D body shape model (STAR) to capture the body surface and a novel part-based 3D skeleton model to capture the bones. We then train a regressor from body shape parameters to skeleton shape parameters and refine the skeleton to satisfy constraints on physical plausibility.
  arXiv  Detail & Related papers  (2022-04-21T14:33:42Z)
• S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling [103.65625425020129]
  We represent the pedestrian's shape, pose and skinning weights as neural implicit functions that are directly learned from data. We demonstrate the effectiveness of our approach on various datasets and show that our reconstructions outperform existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-01-17T02:16:56Z)
• STAR: Sparse Trained Articulated Human Body Regressor [62.71047277943326]
  We introduce STAR, which is quantitatively and qualitatively superior to SMPL. SMPL has a huge number of parameters resulting from its use of global blend shapes. SMPL factors pose-dependent deformations from body shape while, in reality, people with different shapes deform differently. We show that the shape space of SMPL is not rich enough to capture the variation in the human population.
  arXiv  Detail & Related papers  (2020-08-19T16:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.