A Dataset and Evaluation for Complex 4D Markerless Human Motion Capture

• URL: http://arxiv.org/abs/2604.12765v1
• Date: Tue, 14 Apr 2026 14:06:43 GMT
• Title: A Dataset and Evaluation for Complex 4D Markerless Human Motion Capture
• Abstract summary: Marker-based motion capture systems have long been the gold standard for accurate 4D human modeling.<n>We present a new dataset and evaluation for complex 4D markerless human motion capture.
• Score: 4.81005617256125
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Marker-based motion capture (MoCap) systems have long been the gold standard for accurate 4D human modeling, yet their reliance on specialized hardware and markers limits scalability and real-world deployment. Advancing reliable markerless 4D human motion capture requires datasets that reflect the complexity of real-world human interactions. Yet, existing benchmarks often lack realistic multi-person dynamics, severe occlusions, and challenging interaction patterns, leading to a persistent domain gap. In this work, we present a new dataset and evaluation for complex 4D markerless human motion capture. Our proposed MoCap dataset captures both single and multi-person scenarios with intricate motions, frequent inter-person occlusions, rapid position exchanges between similarly dressed subjects, and varying subject distances. It includes synchronized multi-view RGB and depth sequences, accurate camera calibration, ground-truth 3D motion capture from a Vicon system, and corresponding SMPL/SMPL-X parameters. This setup ensures precise alignment between visual observations and motion ground truth. Benchmarking state-of-the-art markerless MoCap models reveals substantial performance degradation under these realistic conditions, highlighting limitations of current approaches. We further demonstrate that targeted fine-tuning improves generalization, validating the dataset's realism and value for model development. Our evaluation exposes critical gaps in existing models and provides a rigorous foundation for advancing robust markerless 4D human motion capture.

Related papers

• Human4K: A Large-Scale 4K Multi-View Mocap Dataset for Whole-Body 3D Human Reconstruction [52.66661102115907]
  We present Human4K, a large-scale 4K multi-view whole-body human reconstruction dataset with mocap-accurate SMPL-X annotations.<n>Human4K contains over six million 4K images captured by an eight-view high-resolution camera system synchronized with a professional Vicon motion capture setup.<n> Experimental results show that training with Human4K consistently improves whole-body reconstruction on standard benchmarks.
  arXiv  Detail & Related papers  (2026-07-15T09:44:25Z)
• High-Fidelity 4D Hand-Object Capture via Multi-View Spatiotemporal Tracking and Physics-Aware Gaussians [38.0162622592412]
  We propose a novel system for the robust and accurate reconstruction of hands and objects from synchronized and calibrated multi-view videos.<n>Our pipeline achieves highly robust, artifact-free reconstruction, providing an efficient foundation for automated 4D asset generation.
  arXiv  Detail & Related papers  (2026-06-14T16:36:49Z)
• Syn4D: A Multiview Synthetic 4D Dataset [92.50132053229817]
  3D reconstruction and tracking of dynamic scenes from monocular video remains an important open challenge in computer vision.<n>To address this limitation, we introduce Syn4D, a multiview synthetic dynamic scenes.<n>Key feature of Syn4D is the ability to unproject any pixel into 3D.
  arXiv  Detail & Related papers  (2026-05-06T17:59:55Z)
• GeoMotion: Rethinking Motion Segmentation via Latent 4D Geometry [61.24189040578178]
  We propose a fully learning-based approach that directly infers moving objects from latent feature representations via attention mechanisms.<n>Our key insight is to bypass explicit correspondence estimation and instead let the model learn to implicitly disentangle object and camera motion.<n>Our approach achieves state-of-the-art motion segmentation performance with high efficiency.
  arXiv  Detail & Related papers  (2026-02-25T11:36:33Z)
• Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding [54.859943475818234]
  We present Motion4D, a novel framework that integrates 2D priors from foundation models into a unified 4D Gaussian Splatting representation.<n>Our method features a two-part iterative optimization framework: 1) Sequential optimization, which updates motion and semantic fields in consecutive stages to maintain local consistency, and 2) Global optimization, which jointly refines all attributes for long-term coherence.<n>Our method significantly outperforms both 2D foundation models and existing 3D-based approaches across diverse scene understanding tasks, including point-based tracking, video object segmentation, and novel view synthesis.
  arXiv  Detail & Related papers  (2025-12-03T09:32:56Z)
• DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modeling [67.95038177144554]
  We introduce DynamicVerse, a physical-scale, multimodal 4D world modeling framework for dynamic real-world video.<n>We employ vision, geometric, and multimodal models to interpret metric-scale static geometry, real-world dynamic motion, instance-level masks, and holistic captions.<n> DynamicVerse delivers a large-scale dataset consisting of 100K+ videos with 800K+ annotated masks and 10M+ frames from internet videos.
  arXiv  Detail & Related papers  (2025-12-02T18:24:27Z)
• Towards High-Consistency Embodied World Model with Multi-View Trajectory Videos [24.111891848073288]
  Embodied world models aim to predict and interact with the physical world through visual observations and actions.<n>MTV-World introduces Multi-view Trajectory-Video control for precise visuomotor prediction.<n>MTV-World achieves precise control execution and accurate physical interaction modeling in complex dual-arm scenarios.
  arXiv  Detail & Related papers  (2025-11-17T02:17:04Z)
• MAMMA: Markerless & Automatic Multi-Person Motion Action Capture [37.06717786024836]
  MAMMA is a markerless motion-capture pipeline that recovers SMPL-X parameters from multi-view video of two-person interaction sequences.<n>We introduce a method that predicts dense 2D surface landmarks conditioned on segmentation masks.<n>We demonstrate that our approach can handle complex person--person interaction and offers greater accuracy than existing methods.
  arXiv  Detail & Related papers  (2025-06-16T02:04:51Z)
• Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera [49.82535393220003]
  Dyn-HaMR is the first approach to reconstruct 4D global hand motion from monocular videos recorded by dynamic cameras in the wild.<n>We show that our approach significantly outperforms state-of-the-art methods in terms of 4D global mesh recovery.<n>This establishes a new benchmark for hand motion reconstruction from monocular video with moving cameras.
  arXiv  Detail & Related papers  (2024-12-17T12:43:10Z)
• MOVIN: Real-time Motion Capture using a Single LiDAR [7.3228874258537875]
  We present MOVIN, the data-driven generative method for real-time motion capture with global tracking. Our framework accurately predicts the performer's 3D global information and local joint details. We implement a real-time application to showcase our method in real-world scenarios.
  arXiv  Detail & Related papers  (2023-09-17T16:04:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.