TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation

• URL: http://arxiv.org/abs/2606.12153v1
• Date: Wed, 10 Jun 2026 14:41:19 GMT
• Title: TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation
• Abstract summary: We introduce TopoCap, the first unified framework capable of extracting motion from monocular video and rigging it onto characters with arbitrary, unseen skeletal topologies.<n>Our key insight is that while skeletal structures are discrete, the underlying physics of motion occupy a continuous, low-dimensional, manifold.<n>We materialize this insight via a two-stage generative pipeline.
• Score: 44.79819257609757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The explosion of generative 3D assets has created a massive demand for animation, yet current motion capture methods remain brittle, restricted to species-specific templates (e.g., SMPL) or requiring labor-intensive manual rigging. We introduce TopoCap, the first unified framework capable of extracting motion from monocular video and retargeting it onto characters with arbitrary, unseen skeletal topologies, i.e., from bipeds to hexapods and inanimate objects, without test-time optimization. Our key insight is that while skeletal structures are combinatorial and discrete, the underlying physics of motion occupy a continuous, low-dimensional manifold. We materialize this insight via a two-stage generative pipeline. First, we learn a Universal Motion Manifold using a Graph CVAE that compresses heterogeneous kinematic chains into a shared, fixed-length latent code. By explicitly conditioning the decoder on a structural embedding of the target rig, we disentangle motion dynamics from skeletal topology. Second, we treat video-to-animation as a conditional flow matching problem, predicting these topology-agnostic codes from visual features. To learn this generalized prior, we introduce Mobjaverse, a massive-scale dataset curated from Objaverse-XL. Comprising over 5,000 unique skeletal topologies and 2 million frames, it exceeds the structural diversity of existing datasets by two orders of magnitude. Extensive experiments demonstrate that \MethodMotion outperforms specialist models on human and quadruped benchmarks while enabling zero-shot retargeting for the long tail of 3D creatures. Dataset is publicly available at https://huggingface.co/datasets/duckduckplz/Mobjaverse.

Related papers

• UniMate: One Unified Model to Animate Diverse Skeletons [22.957889325110912]
  We present UniMate, a foundation model that synthesizes articulated motion for arbitrary skeletons from a rigged 3D asset and a text prompt.<n>UniMate introduces a topology-aware diffusion transformer, which integrates skeletal topology into attention.<n>We also curate UniML3D, 13,006 motion sequences spanning bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid objects.
  arXiv  Detail & Related papers  (2026-09-04T17:59:00Z)
• MotionDreamer: Universal Skeletal Motion Generation for 3D Rigged Shapes [52.45449797329548]
  MotionDreamer is a diffusion-based framework for category-agnostic skeletal animation generation from 2D video guidance.<n>We propose a structural-semantic injection mechanism to bridge the gap between 2D visual motion cues and heterogeneous 3D skeletal structures.<n>Our approach significantly outperforms existing methods, setting a new state-of-the-art benchmark for robust and efficient 4D asset generation.
  arXiv  Detail & Related papers  (2026-06-01T00:42:31Z)
• ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes [55.32681167870698]
  Video-informed Pose Spaces (ViPS) is a feed-forward framework that discovers the latent distribution of valid articulations for auto-rigged meshes.<n>ViPS transfers generative video priors into a universal distribution over a given rig parameterization.<n>Our evaluations show that ViPS matches the performance of state-of-the-art methods trained on synthetic artist-created 4D data in both plausibility and diversity.
  arXiv  Detail & Related papers  (2026-04-19T21:21:11Z)
• Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding [50.098085774845195]
  We propose a paradigm shift by leveraging the implicit spatial prior within large-scale generation models.<n>We introduce GeneEGA-3D (Video Extracted Generative Awareness), a plug-and-play framework that repurposes a pre-trained video diffusion model as a Latent World Simulator.
  arXiv  Detail & Related papers  (2026-03-19T17:59:58Z)
• Controllable Complex Human Motion Video Generation via Text-to-Skeleton Cascades [32.12940211414945]
  We propose a two-stage cascaded framework that addresses both limitations.<n>An autoregressive text-to-skeleton model generates 2D pose sequences from natural language descriptions.<n>A pose-conditioned video diffusion model synthesizes videos from a reference image and the generated skeleton sequence.
  arXiv  Detail & Related papers  (2026-03-09T07:04:29Z)
• Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation [32.57062686780495]
  Superman is a unified framework that bridges visual perception with temporal, skeleton-based motion generation.<n>This module flexibly processes diverse, temporal inputs, unifying 3D skeleton pose estimation from video (perception) with skeleton-based motion prediction and in-betweening (generation)
  arXiv  Detail & Related papers  (2026-02-02T17:59:01Z)
• MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videos [31.168481928653748]
  MoCapAnything is a reference-guided, factorized framework for 3D motion capture.<n>It reconstructs a rotation-based animation that directly drives the specific asset.<n>It delivers high-quality skeletal animations and meaningful cross-species animations.
  arXiv  Detail & Related papers  (2025-12-11T18:09:48Z)
• UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework [54.337290937468175]
  We propose UniMo, an autoregressive model for joint modeling of 2D human videos and 3D human motions within a unified framework.<n>We show that our method simultaneously generates corresponding videos and motions while performing accurate motion capture.
  arXiv  Detail & Related papers  (2025-12-03T16:03:18Z)
• SMF: Template-free and Rig-free Animation Transfer using Kinetic Codes [32.324844649352166]
  Animation retargetting applies sparse motion description to a character mesh to produce a semantically plausible and temporally coherent full-body sequence.<n>We propose Self-supervised Motion Fields (SMF), a self-supervised framework that is trained with only sparse motion representations.<n>Our architecture comprises dedicated spatial and temporal gradient predictors, which are jointly trained in an end-to-end fashion.
  arXiv  Detail & Related papers  (2025-04-07T08:42:52Z)
• AnyTop: Character Animation Diffusion with Any Topology [54.07731933876742]
  We introduce AnyTop, a diffusion model that generates motions for diverse characters with distinct motion dynamics.<n>Our work features a transformer-based denoising network, tailored for arbitrary skeleton learning.<n>Our evaluation demonstrates that AnyTops well, even with as few as three training examples per topology, and can produce motions for unseen skeletons as well.
  arXiv  Detail & Related papers  (2025-02-24T17:00:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.