Dancing Stick Figures: An Introductory Dataset for Training Video Generation Models
- URL: http://arxiv.org/abs/2608.29123v1
- Date: Sat, 29 Aug 2026 08:03:19 GMT
- Title: Dancing Stick Figures: An Introductory Dataset for Training Video Generation Models
- Abstract summary: Dancing Stick Figures is a synthetic video dataset built against three obstacles.<n>For speed, its 64x64, 64-frame reference task is sized for practical repeated training on a single workstation GPU.<n>For accessibility, the release is a 0.79-GB training tier of 4,020 video clips--1,340 six-second source motions.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Training a video-generation model from scratch is hard for reasons that precede model design. The feedback loop is long: a failure that appears only after a training run can make each attempted fix another run. The data are hard to reach: the corpora and recipes behind strong models are large, heterogeneous, and often unreleased. And scoring is blunt: open-ended generation has no single correct output, and an aggregate score does not by itself establish whether a sample succeeds or which property failed. Dancing Stick Figures is a synthetic video dataset built against these three obstacles. For iteration speed, its 64x64, 64-frame reference task is sized for practical repeated training on a single workstation GPU. For accessibility, the release is a 0.79-GB training tier of 4,020 video clips--1,340 six-second source motions, each rendered from three cameras by a deterministic dataset-generation harness--with checkpoints and a Colab workflow that reruns the reference training pipeline at reduced budget on a 16 GB Tesla T4. For scoring, every frame retains its generating state (ARDY cskel27 joint positions, camera, body parameters, and source motion) and per-pixel depth, surface normals, and part labels. These annotations support dataset-specific metrics for visible topology and part-wise motion; corruptions expose their sensitivities and blind spots.
Related papers
- Manifold4D: Denoising on Point Cloud Rendered Manifolds for Video Re-shooting [16.163736801043942]
Video re-shooting re-renders a monocular video of a dynamic scene along a user-specified camera trajectory.<n>The dominant recipe supplies the target geometry explicitly.<n>We propose MANIFOLD4D, which injects the render directly into the initial noise of flow matching.
arXiv Detail & Related papers (2026-08-28T10:39:52Z) - DreamHand: Repurposing Video Diffusion Models for Occlusion-Robust Egocentric 3D Hand Motion Recovery [51.92897182932217]
Egocentric video offers scalable manipulation data for embodied AI.<n>However, recovering metric 3D hand trajectories is challenging due to severe object occlusion and frequent out-of-sight gaps.<n>We introduce DreamHand, an offline clip-level framework that extracts features via a Deterministic Clean-Latent and decodes them with a Bidirectional Spatiotemporal Decoder.
arXiv Detail & Related papers (2026-08-20T17:46:24Z) - RigidBench: Evaluating Rigid-Body Physics in Video Generation Models [16.53417077740312]
We introduce a simulator-grounded benchmark that compares a generated continuation with a reference rollout from the same initial frame and motion description.<n>We evaluate eight models on the same 100 examples with ten measurements that keep these aspects separate.<n>Full fine-tuning reduces 3D trajectory error by about 20% with almost no change in SSIM.
arXiv Detail & Related papers (2026-08-16T06:05:20Z) - What-If World: A Causal Benchmark for General World Models in Embodied Scenarios [23.527918480081013]
Video generation models are increasingly used as world simulators for tasks like driving and robotic manipulation.<n>What matters in these settings is not whether a single video looks right, but whether the model's output changes when its input changes.<n>We test this by giving a model two prompts describing the same scene with one physical detail varied, and checking whether the two videos diverge the way physics predicts.
arXiv Detail & Related papers (2026-05-26T19:02:26Z) - Geo-Align: Video Generation Alignment via Metric Geometry Reward [53.65904111864641]
Geo-Align is the first Reinforcement Learning framework specifically designed for camera-controlled video re-rendering.<n>Built upon a pretrained model, we optimize the model through a scale-aware perceptual reward mechanism.<n>Experiments demonstrate that Geo-Align consistently outperforms existing supervised learning baselines in both precise camera controllability and visual fidelity.
arXiv Detail & Related papers (2026-05-22T17:59:43Z) - Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Video [81.44600627066747]
Mesh4D is a feed-forward model for monocular 4D mesh reconstruction.<n>Our key contribution is a compact latent space that encodes the entire animation sequence in a single pass.<n>We evaluate Mesh4D on reconstruction and view novel benchmarks, outperforming prior methods in recovering accurate 3D shape and deformation.
arXiv Detail & Related papers (2026-01-08T18:59:56Z) - Trace Anything: Representing Any Video in 4D via Trajectory Fields [98.85848134960172]
Trajectory Field is a dense mapping that assigns a continuous 3D trajectory function of time to each pixel in every frame.<n>We introduce Trace Anything, a neural network that predicts the entire trajectory field in a single feed-forward pass.<n>We trained the Trace Anything model on large-scale 4D data, including data from our new platform.
arXiv Detail & Related papers (2025-10-15T17:59:04Z) - Restage4D: Reanimating Deformable 3D Reconstruction from a Single Video [56.781766315691854]
We introduce textbfRestage4D, a geometry-preserving pipeline for video-conditioned 4D restaging.<n>We validate Restage4D on DAVIS and PointOdyssey, demonstrating improved geometry consistency, motion quality, and 3D tracking performance.
arXiv Detail & Related papers (2025-08-08T21:31:51Z) - Learning to Track Any Points from Human Motion [55.831218129679144]
We propose an automated pipeline to generate pseudo-labeled training data for point tracking.<n>A point tracking model trained on AnthroTAP achieves annotated state-of-the-art performance on the TAP-Vid benchmark.
arXiv Detail & Related papers (2025-07-08T17:59:58Z) - Direct Motion Models for Assessing Generated Videos [38.04485796547767]
A current limitation of video generative video models is that they generate plausible looking frames, but poor motion.<n>Here we go beyond FVD by developing a metric which better measures plausible object interactions and motion.<n>We show that using point tracks instead of pixel reconstruction or action recognition features results in a metric which is markedly more sensitive to temporal distortions in synthetic data.
arXiv Detail & Related papers (2025-04-30T22:34:52Z) - VideoLifter: Lifting Videos to 3D with Fast Hierarchical Stereo Alignment [54.66217340264935]
VideoLifter is a novel video-to-3D pipeline that leverages a local-to-global strategy on a fragment basis.<n>It significantly accelerates the reconstruction process, reducing training time by over 82% while holding better visual quality than current SOTA methods.
arXiv Detail & Related papers (2025-01-03T18:52:36Z) - Masked Motion Encoding for Self-Supervised Video Representation Learning [84.24773072241945]
We present Masked Motion MME, a new pre-training paradigm that reconstructs both appearance and motion information to explore temporal clues.
Motivated by the fact that human is able to recognize an action by tracking objects' position changes and shape changes, we propose to reconstruct a motion trajectory that represents these two kinds of change in the masked regions.
Pre-trained with our MME paradigm, the model is able to anticipate long-term and fine-grained motion details.
arXiv Detail & Related papers (2022-10-12T11:19:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.