OpenCap Monocular: 3D Human Kinematics and Musculoskeletal Dynamics from a Single Smartphone Video

• URL: http://arxiv.org/abs/2603.24733v1
• Date: Wed, 25 Mar 2026 18:56:46 GMT
• Title: OpenCap Monocular: 3D Human Kinematics and Musculoskeletal Dynamics from a Single Smartphone Video
• Abstract summary: Quantifying human movement (kinematics) and musculoskeletal forces (kinetics) at scale could transform prediction, treatment, and monitoring of mobility-related conditions.<n>We introduce OpenCap Monocular, an algorithm that estimates 3D skeletal kinematics and kinetics from a single smartphone video.
• Score: 0.2012293145190529
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Quantifying human movement (kinematics) and musculoskeletal forces (kinetics) at scale, such as estimating quadriceps force during a sit-to-stand movement, could transform prediction, treatment, and monitoring of mobility-related conditions. However, quantifying kinematics and kinetics traditionally requires costly, time-intensive analysis in specialized laboratories, limiting clinical translation. Scalable, accurate tools for biomechanical assessment are needed. We introduce OpenCap Monocular, an algorithm that estimates 3D skeletal kinematics and kinetics from a single smartphone video. The method refines 3D human pose estimates from a monocular pose estimation model (WHAM) via optimization, computes kinematics of a biomechanically constrained skeletal model, and estimates kinetics via physics-based simulation and machine learning. We validated OpenCap Monocular against marker-based motion capture and force plate data for walking, squatting, and sit-to-stand tasks. OpenCap Monocular achieved low kinematic error (4.8° mean absolute error for rotational degrees of freedom; 3.4 cm for pelvis translations), outperforming a regression-only computer vision baseline by 48% in rotational accuracy (p = 0.036) and 69% in translational accuracy (p < 0.001). OpenCap Monocular also estimated ground reaction forces during walking with accuracy comparable to, or better than, our prior two-camera OpenCap system. We demonstrate that the algorithm estimates important kinetic outcomes with clinically meaningful accuracy in applications related to frailty and knee osteoarthritis, including estimating knee extension moment during sit-to-stand transitions and knee adduction moment during walking. OpenCap Monocular is deployed via a smartphone app, web app, and secure cloud computing (https://opencap.ai), enabling free, accessible single-smartphone biomechanical assessments.

Related papers

• MotionVLA: Vision-Language-Action Model for Humanoid Motion [54.785960777274276]
  We present DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams.<n>We also present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence.
  arXiv  Detail & Related papers  (2026-06-13T06:10:48Z)
• SAM4Dcap: Training-free Biomechanical Twin System from Monocular Video [16.202462062701038]
  SAM4Dcap is an open-source framework for estimating biomechanical metrics from monocular video without additional training.<n>The pipeline converts reconstructed meshes into trajectory files compatible with diverse musculoskeletal models.<n>Preliminary evaluations on walking and drop-jump tasks indicate that SAM4Dcap has the potential to achieve knee kinematic predictions comparable to multi-view systems.
  arXiv  Detail & Related papers  (2026-02-14T13:09:38Z)
• MonoMSK: Monocular 3D Musculoskeletal Dynamics Estimation [9.3102957142944]
  We introduce MonoMSK, a hybrid framework that bridges data-driven learning and physics-based simulation.<n>MonoMSK jointly recovers both kinematics (motions) and kinetics (forces and torques) through an anatomically accurate musculoskeletal model.<n> Experiments on BML-MoVi, BEDLAM, and OpenCap show that MonoMSK significantly outperforms state-of-the-art methods in kinematic accuracy.
  arXiv  Detail & Related papers  (2025-11-24T17:20:17Z)
• Validation of Human Pose Estimation and Human Mesh Recovery for Extracting Clinically Relevant Motion Data from Videos [79.62407455005561]
  Marker-less motion capture using human pose estimation produces results in-line with the results of both the IMU and MoCap kinematics.<n>While there is still room for improvement when it comes to the quality of the data produced, we believe that this compromise is within the room of error.
  arXiv  Detail & Related papers  (2025-03-18T22:18:33Z)
• Estimating Body and Hand Motion in an Ego-sensed World [62.61989004520802]
  We present EgoAllo, a system for human motion estimation from a head-mounted device.<n>Using only egocentric SLAM poses and images, EgoAllo guides sampling from a conditional diffusion model to estimate 3D body pose, height, and hand parameters.
  arXiv  Detail & Related papers  (2024-10-04T17:59:57Z)
• Comparison of Kinematics and Kinetics Between OpenCap and a Marker-Based Motion Capture System in Cycling [0.0]
  This study evaluates the agreement of marker-based and markerless (OpenCap) motion capture systems in assessing joint kinematics and kinetics during cycling.<n>Key metrics, including joint angles, moments, and joint reaction loads, were computed using OpenSim.<n>Results revealed very strong agreement (r GT 0.9) for hip (flexion/extension), knee (flexion/extension), and ankle (dorsiflexion/plantarflexion) joint angles.
  arXiv  Detail & Related papers  (2024-08-20T15:57:40Z)
• OpenCapBench: A Benchmark to Bridge Pose Estimation and Biomechanics [2.2188004943970627]
  We develop OpenCapBench to offer an easy-to-use unified benchmark to assess common tasks in human pose estimation. OpenCapBench computes consistent kinematic metrics through joints angles provided by an open-source musculoskeletal modeling software (OpenSim) We introduce SynthPose, a new approach that enables finetuning of pre-trained 2D human pose models to predict an arbitrarily denser set of keypoints for accurate kinematic analysis.
  arXiv  Detail & Related papers  (2024-06-14T07:37:28Z)
• 3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data [4.130944152992895]
  We propose a novel biomechanics-aware network that directly outputs 3D kinematics from two input views. Our experiments demonstrate that the proposed approach, only trained on synthetic data, outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-20T17:33:40Z)
• Towards Single Camera Human 3D-Kinematics [15.559206592078425]
  We propose a novel approach for direct 3D human kinematic estimation D3KE from videos using deep neural networks. Our experiments demonstrate that the proposed end-to-end training is robust and outperforms 2D and 3D markerless motion capture based kinematic estimation pipelines.
  arXiv  Detail & Related papers  (2023-01-13T08:44:09Z)
• Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening [59.88594294676711]
  Modern deep learning based motion synthesis approaches barely consider the physical plausibility of synthesized motions. We propose a system Skeleton2Humanoid'' which performs physics-oriented motion correction at test time. Experiments on the challenging LaFAN1 dataset show our system can outperform prior methods significantly in terms of both physical plausibility and accuracy.
  arXiv  Detail & Related papers  (2022-10-09T16:15:34Z)
• Neural Monocular 3D Human Motion Capture with Physical Awareness [76.55971509794598]
  We present a new trainable system for physically plausible markerless 3D human motion capture. Unlike most neural methods for human motion capture, our approach is aware of physical and environmental constraints. It produces smooth and physically principled 3D motions in an interactive frame rate in a wide variety of challenging scenes.
  arXiv  Detail & Related papers  (2021-05-03T17:57:07Z)
• PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time [89.68248627276955]
  Marker-less 3D motion capture from a single colour camera has seen significant progress. However, it is a very challenging and severely ill-posed problem. We present PhysCap, the first algorithm for physically plausible, real-time and marker-less human 3D motion capture.
  arXiv  Detail & Related papers  (2020-08-20T10:46:32Z)
• Contact and Human Dynamics from Monocular Video [73.47466545178396]
  Existing deep models predict 2D and 3D kinematic poses from video that are approximately accurate, but contain visible errors. We present a physics-based method for inferring 3D human motion from video sequences that takes initial 2D and 3D pose estimates as input.
  arXiv  Detail & Related papers  (2020-07-22T21:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.