BioHuman: Learning Biomechanical Human Representations from Video

• URL: http://arxiv.org/abs/2605.14772v1
• Date: Thu, 14 May 2026 12:36:53 GMT
• Title: BioHuman: Learning Biomechanical Human Representations from Video
• Abstract summary: We introduce a simulation-based framework for estimating muscle activations from existing motion capture datasets.<n>We propose BioHuman, an end-to-end model that takes monocular video as input and jointly predicts human motion and muscle activations.
• Score: 11.653644899302542
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding human motion beyond surface kinematics is crucial for motion analysis, rehabilitation, and injury risk assessment. However, progress in this domain is limited by the lack of large-scale datasets with biomechanical annotations, and by existing approaches that cannot directly infer internal biomechanical states from visual observations. In this paper, we introduce a simulation-based framework for estimating muscle activations from existing motion capture datasets, resulting in BioHuman10M, a large-scale dataset with synchronized video, motion, and activations. Building on BioHuman10M, we propose BioHuman, an end-to-end model that takes monocular video as input and jointly predicts human motion and muscle activations, effectively bridging visual observations and internal biomechanical states. Extensive experiments demonstrate that BioHuman enables accurate reconstruction of both kinematic motion and muscle activity, and generalizes across diverse subjects and motions. We believe our approach establishes a new benchmark for video-based biomechanical understanding and opens up new possibilities for physically grounded human modeling.

Related papers

• Pose-to-Biomechanics: Bridging 3D Human Pose Estimation and Biomechanical Attribute Prediction [4.726777092009554]
  BioModule is a lightweight plug-in temporal transformer that attaches downstream of any 3D pose estimator.<n>BioModule is estimator-agnostic and requires no modification of the upstream pose model.<n>We benchmark BioModule across seven state-of-the-art 3D pose estimators.
  arXiv  Detail & Related papers  (2026-07-09T17:31:16Z)
• SOMA: From Surface Observations to Muscle Anatomy [55.23576061150391]
  We present SOMA, a person-specific model that infers bio-temporal muscle behavior from surface signals obtained using RGB cameras.<n>We show how our method provides anatomically grounded animations without the complexity of traditional simulations.
  arXiv  Detail & Related papers  (2026-06-08T09:20:46Z)
• Towards World Models in Biomedical Research [77.59710966764466]
  We propose biomedical world models as a paradigm for AI-driven discovery.<n>These models learn latent representations of molecular, cellular, tissue and clinical states, together with intervention-conditioned dynamics.<n>We discuss how biomedical world models could function as data engines, environment simulators and scientific planning substrates across applications including virtual cells, organoids, virtual patients and surgical simulation.
  arXiv  Detail & Related papers  (2026-06-04T09:28:54Z)
• HumanScore: Benchmarking Human Motions in Generated Videos [16.29197573772623]
  We present HumanScore, a framework to evaluate the quality of human motions in AI-generated videos.<n>HumanScore defines six interpretable metrics spanning kinematic plausibility, temporal stability, and biomechanical consistency.<n>Our analysis reveals consistent gaps between perceptual plausibility and motion biomechanical fidelity.
  arXiv  Detail & Related papers  (2026-04-22T03:51:19Z)
• Towards Motion Turing Test: Evaluating Human-Likeness in Humanoid Robots [54.54929302887405]
  We present the Human-Humanoid Motion dataset, which consists of 1,000 motion sequences spanning 15 action categories.<n>All motion sequences are converted into SMPL-X representations to eliminate the influence of visual appearance.<n>Analysis of the collected data reveals that humanoid motions still exhibit noticeable deviations from human movements.
  arXiv  Detail & Related papers  (2026-03-06T11:40:56Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• KinTwin: Imitation Learning with Torque and Muscle Driven Biomechanical Models Enables Precise Replication of Able-Bodied and Impaired Movement from Markerless Motion Capture [2.44755919161855]
  High-quality movement analysis could greatly benefit movement science and rehabilitation.<n>We show the potential for using imitation learning to enable high-quality movement analysis in clinical practice.
  arXiv  Detail & Related papers  (2025-05-19T17:58:03Z)
• Reinforcement learning-based motion imitation for physiologically plausible musculoskeletal motor control [47.423243831156285]
  We present a model-free motion imitation framework (KINESIS) to advance the understanding of muscle-based motor control.<n>We demonstrate that KINESIS achieves strong imitation performance on 1.9 hours of motion capture data.<n>KINESIS generates muscle activity patterns that correlate well with human EMG activity.
  arXiv  Detail & Related papers  (2025-03-18T18:37:49Z)
• Biomechanics-Guided Residual Approach to Generalizable Human Motion Generation and Estimation [21.750804738752105]
  We propose BioVAE, a biomechanics-aware framework with three core innovations.<n>We show that BioVAE achieves state-of-the-art performance on multiple benchmarks.
  arXiv  Detail & Related papers  (2025-03-08T10:22:36Z)
• Muscles in Time: Learning to Understand Human Motion by Simulating Muscle Activations [64.98299559470503]
  Muscles in Time (MinT) is a large-scale synthetic muscle activation dataset. It contains over nine hours of simulation data covering 227 subjects and 402 simulated muscle strands. We show results on neural network-based muscle activation estimation from human pose sequences.
  arXiv  Detail & Related papers  (2024-10-31T18:28:53Z)
• MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints [50.61346764110482]
  We integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We also propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron network.
  arXiv  Detail & Related papers  (2024-04-16T02:18:18Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• 3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data [4.130944152992895]
  We propose a novel biomechanics-aware network that directly outputs 3D kinematics from two input views. Our experiments demonstrate that the proposed approach, only trained on synthetic data, outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-20T17:33:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.