Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation

• URL: http://arxiv.org/abs/2609.01840v1
• Date: Tue, 01 Sep 2026 20:17:00 GMT
• Title: Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation
• Abstract summary: We perform cross-model distillation from the Sapiens 2 pose model into the SAM 3D Body model.<n>Fine-tuning improves same-view 2D keypoint agreement with the Sapiens reference.
• Score: 1.0195618602298682
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Spontaneous movement is one of the earliest windows onto an infant's neuromotor health, and structured clinical instruments that score it are validated early predictors of cerebral-palsy risk. However, they require specially trained raters, are time-consuming, and carry inter-rater variability. This motivates automated, video-based markerless assessment, especially as marker-based motion capture is impractical in infants. Yet the foundation models that make markerless capture possible are trained almost entirely on adults: our recent multi-view infant study found that no single model is jointly best, with strong 2D keypoint accuracy and direct 3D body recovery split across different models. While that study identifies this trade-off, it does not resolve it. Here, we perform cross-model distillation from the Sapiens 2 pose model into the SAM 3D Body model, using unannotated infant video alone. A frozen teacher supplies dense pseudo-labels, and a differentiable renderer aligns the predicted mesh to them in the training loop. On eleven held-out infants (18 sessions, 173 recordings) under our prior study's multi-view protocol, fine-tuning improves same-view 2D keypoint agreement with the Sapiens reference (median body percentage of correct keypoints @ 10px 0.22 -> 0.42, face 0.22 -> 0.42) and Procrustes-aligned mean per joint 3D position error (25.5 -> 22.2 mm). This demonstrates how cross-model distillation improves SAM 3D Body model performance on infants.

Related papers

• Markerless Motion Capture for Biomechanical Whole-Body Kinematic Estimation in Infants [0.9332987715848714]
  This study systematically evaluated three state-of-the-art pose estimation frameworks on 100 videos over 13 sessions of 8 infants recorded with a multi-view motion capture system.<n>We quantified keypoint detection accuracy using reprojection error, geometric consistency, and Procrustes-aligned 3D position error, and demonstrated proof-of-concept for fitting an inverse kinematic framework to infant data.<n>We demonstrate in a case comparison example that biomechanical models fit to SAM 3D estimates distinguish representative movement patterns in infants related to motor development, as identified by a clinical expert.
  arXiv  Detail & Related papers  (2026-05-16T18:55:06Z)
• Reconstructing Humans with a Biomechanically Accurate Skeleton [55.06027148976482]
  We introduce a method for reconstructing 3D humans from a single image using a biomechanically accurate skeleton model.<n>Compared to state-of-the-art methods for 3D human mesh recovery, our model achieves competitive performance on standard benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T17:56:24Z)
• BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos [6.280386490530478]
  BioPose is a learning-based framework for predicting biomechanically accurate 3D human pose directly from monocular videos.<n>It includes a Multi-Query Human Mesh Recovery model (MQ-HMR), a Neural Inverse Kinematics (NeurIK) model, and a 2D-informed pose refinement technique.<n> Experiments on benchmark datasets demonstrate that BioPose significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-14T02:56:19Z)
• Extending 3D body pose estimation for robotic-assistive therapies of autistic children [4.751886527142779]
  We develop a 3D pose estimator for children with Autism. Our method has an error below $0.3m$, which is considered acceptable for this kind of application. In real-world settings, the proposed model performs similarly to a Kinect depth camera.
  arXiv  Detail & Related papers  (2024-02-12T19:11:03Z)
• Semi-Supervised 2D Human Pose Estimation Driven by Position Inconsistency Pseudo Label Correction Module [74.80776648785897]
  The previous method ignored two problems: (i) When conducting interactive training between large model and lightweight model, the pseudo label of lightweight model will be used to guide large models. We propose a semi-supervised 2D human pose estimation framework driven by a position inconsistency pseudo label correction module (SSPCM) To further improve the performance of the student model, we use the semi-supervised Cut-Occlude based on pseudo keypoint perception to generate more hard and effective samples.
  arXiv  Detail & Related papers  (2023-03-08T02:57:05Z)
• Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation [63.199549837604444]
  3D human pose estimation approaches leverage different forms of strong (2D/3D pose) or weak (multi-view or depth) paired supervision. We cast 3D pose learning as a self-supervised adaptation problem that aims to transfer the task knowledge from a labeled source domain to a completely unpaired target. We evaluate different self-adaptation settings and demonstrate state-of-the-art 3D human pose estimation performance on standard benchmarks.
  arXiv  Detail & Related papers  (2022-04-05T03:52:57Z)
• A Spatio-temporal Attention-based Model for Infant Movement Assessment from Videos [44.71923220732036]
  We develop a new method for fidgety movement assessment using human poses extracted from short clips. Human poses capture only relevant motion profiles of joints and limbs and are free from irrelevant appearance artifacts. Our experiments show that the proposed method achieves the ROC-AUC score of 81.87%, significantly outperforming existing competing methods with better interpretability.
  arXiv  Detail & Related papers  (2021-05-20T14:31:54Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)
• Synthetic Training for Monocular Human Mesh Recovery [100.38109761268639]
  This paper aims to estimate 3D mesh of multiple body parts with large-scale differences from a single RGB image. The main challenge is lacking training data that have complete 3D annotations of all body parts in 2D images. We propose a depth-to-scale (D2S) projection to incorporate the depth difference into the projection function to derive per-joint scale variants.
  arXiv  Detail & Related papers  (2020-10-27T03:31:35Z)
• Invariant Representation Learning for Infant Pose Estimation with Small Data [14.91506452479778]
  We release a hybrid synthetic and real infant pose dataset with small yet diverse real images as well as generated synthetic infant poses. In our ablation study, with identical network structure, models trained on SyRIP dataset show noticeable improvement over the ones trained on the only other public infant pose datasets. One of our best infant pose estimation performers on the state-of-the-art DarkPose model shows mean average precision (mAP) of 93.6.
  arXiv  Detail & Related papers  (2020-10-13T01:10:14Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.