MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons

• URL: http://arxiv.org/abs/2604.28130v1
• Date: Thu, 30 Apr 2026 17:16:38 GMT
• Title: MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons
• Abstract summary: We present the first fully end-to-end framework in which Videoto-Pose and Pose-to-Rotation are learnable and optimized jointly.<n>We show that our method reduces rotation error from 17 degrees to 10 degrees, and to 6.54 degrees on unseen skeletons, while achieving 20x faster inference than mesh-based pipelines.
• Score: 56.68975315643491
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent methods for arbitrary-skeleton motion capture from monocular video follow a factorized pipeline, where a Video-to-Pose network predicts joint positions and an analytical inverse-kinematics (IK) stage recovers joint rotations. While effective, this design is inherently limited, since joint positions do not fully determine rotations and leave degrees of freedom such as bone-axis twist ambiguous, and the non-differentiable IK stage prevents the system from adapting to noisy predictions or optimizing for the final animation objective. In this work, we present the first fully end-to-end framework in which both Video-to-Pose and Pose-to-Rotation are learnable and jointly optimized. We observe that the ambiguity in pose-to-rotation mapping arises from missing coordinate system information: the same joint positions can correspond to different rotations under different rest poses and local axis conventions. To resolve this, we introduce a reference pose-rotation pair from the target asset, which, together with the rest pose, not only anchors the mapping but also defines the underlying rotation coordinate system. This formulation turns rotation prediction into a well-constrained conditional problem and enables effective learning. In addition, our model predicts joint positions directly from video without relying on mesh intermediates, improving both robustness and efficiency. Both stages share a skeleton-aware Global-Local Graph-guided Multi-Head Attention (GL-GMHA) module for joint-level local reasoning and global coordination. Experiments on Truebones Zoo and Objaverse show that our method reduces rotation error from ~17 degrees to ~10 degrees, and to 6.54 degrees on unseen skeletons, while achieving ~20x faster inference than mesh-based pipelines. Project page: https://animotionlab.github.io/MoCapAnythingV2/

Related papers

• RASA: Disentangled Spatial-Motional Priors for Cross-Identity Character Animation [85.41886214825894]
  Cross-identity character animation aims to drive a target identity from a reference image to follow the motion of a source character from a driving video.<n>We introduce Reference-Aware Structural Alignment (RASA), a framework that disentangles spatial mapping from motion control.<n>Our work establishes a new paradigm showing disentangled spatial and motional priors are key to robust character animation.
  arXiv  Detail & Related papers  (2026-08-28T11:36:44Z)
• Amortized Inverse Kinematics via Graph Attention for Real-Time Human Avatar Animation [16.714345824701386]
  Inverse kinematics is a core operation in animation, robotics, and biomechanics.<n>We introduce IK-GAT, a lightweight graph-attention network that reconstructs full-body joint orientations from 3D joint positions in a single forward pass.<n>IK-GAT produces animation-ready local rotations that can directly drive a rigged avatar or be converted to pose parameters of SMPL-like body models.
  arXiv  Detail & Related papers  (2026-04-17T18:30:20Z)
• End-to-End Multi-Person Pose Estimation with Pose-Aware Video Transformer [7.19764062839405]
  We present a fully end-to-end framework for multi-person 2D pose estimation in videos.<n>A key challenge is to associate individuals across frames under complex and overlapping temporal trajectories.<n>We introduce a novel Pose-Aware VideoErEr Network (PAVE-Net), which features a spatial encoder to model intra-frame relations and atemporal decoder pose.
  arXiv  Detail & Related papers  (2025-11-17T10:19:35Z)
• PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Control [67.17998939712326]
  We present PoseDiff, a conditional diffusion model that unifies robot state estimation and control within a single framework.<n>At its core, PoseDiff maps raw visual observations into structured robot states-such as 3D keypoints or joint angles-from a single RGB image.<n>Building upon this foundation, PoseDiff extends naturally to video-to-action inverse dynamics.
  arXiv  Detail & Related papers  (2025-09-29T10:55:48Z)
• DeLiVR: Differential Spatiotemporal Lie Bias for Efficient Video Deraining [21.816338275013702]
  We propose DeLiVR, an efficient video deraining method that injects Lie-group differential biases directly into attention scores of the network.<n>A rotation-bounded Lie relative bias predicts the in-plane angle of each frame using a compact prediction module.<n>A differential group displacement computes angular differences between frames adjacent to estimate a velocity.<n>This bias combines temporal decay and attention masks to focus on inter-frame relationships while precisely matching the direction of rain streaks.
  arXiv  Detail & Related papers  (2025-09-26T00:29:36Z)
• PoseGRAF: Geometric-Reinforced Adaptive Fusion for Monocular 3D Human Pose Estimation [5.223657684081615]
  Existing monocular 3D pose estimation methods rely on joint positional features, while overlooking intrinsic directional and angular correlations within the skeleton.<n>We propose the PoseGRAF framework to address these challenges.<n> Experimental results on the Human3.6M and MPI-INF-3DHP datasets show that our method exceeds state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-06-17T14:59:56Z)
• PoseGraphNet++: Enriching 3D Human Pose with Orientation Estimation [43.261111977510105]
  Existing skeleton-based 3D human pose estimation methods only predict joint positions. We present PoseGraphNet++, a novel 2D-to-3D lifting Graph Convolution Network that predicts the complete human pose in 3D.
  arXiv  Detail & Related papers  (2023-08-22T13:42:15Z)
• Shuffled Autoregression For Motion Interpolation [53.61556200049156]
  This work aims to provide a deep-learning solution for the motion task. We propose a novel framework, referred to as emphShuffled AutoRegression, which expands the autoregression to generate in arbitrary (shuffled) order. We also propose an approach to constructing a particular kind of dependency graph, with three stages assembled into an end-to-end spatial-temporal motion Transformer.
  arXiv  Detail & Related papers  (2023-06-10T07:14:59Z)
• Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation [59.94032196768748]
  We propose a relative information encoding method that yields positional and temporal enhanced representations. Our method outperforms state-of-the-art methods on two public datasets.
  arXiv  Detail & Related papers  (2021-07-29T14:12:19Z)
• MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency [72.82534577726334]
  We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video. Our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation.
  arXiv  Detail & Related papers  (2020-06-22T08:50:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.