Kirin: Animal Motion Generation from In-the-Wild Video

• URL: http://arxiv.org/abs/2609.01823v1
• Date: Tue, 01 Sep 2026 19:56:15 GMT
• Title: Kirin: Animal Motion Generation from In-the-Wild Video
• Abstract summary: Kirin is a framework that reconstructs motion from video, learns motion priors at scale, and generates realistic motion that can be directly applied to animated assets.<n>AiM3D is the first large-scale dataset offering aligned video-text-motions for quadruped animals.<n>By leveraging an off-the-shelf image-to-3D model, we automatically rig and animate 3D meshes using generated motion, producing ready-to-render animated animals.
• Score: 40.9434764171182
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding animal motion is fundamental to modeling animal behavior and biomechanics, yet progress in this area lags far behind human motion research due to the scarcity of high-quality motion data. While human motion can be captured in controlled environments, it is impractical for most animal species, resulting in small, domain-limited datasets that restrict downstream applications such as animation. To address this challenge, we introduce Kirin, a framework that reconstructs motion from video, learns motion priors at scale, and generates realistic motion that can be directly applied to animated assets. Using large collections of in-the-wild animal videos, we reconstruct 3D motion sequences and pair them with captions to create AiM3D, the first large-scale dataset offering aligned video-text-motion tuples for quadruped animals. Building on this dataset, we develop a visual-guided motion generation model that conditions on both text and image to guide the generation of realistic motion across diverse animal species. Finally, by leveraging an off-the-shelf image-to-3D model, we automatically rig and animate 3D meshes using generated motion, producing ready-to-render animated animals. Together, our dataset and framework establish a new foundation for large-scale, text and image conditioned animal motion generation and animation. Project page: https://kirin-ani.github.io/.

Related papers

• 3D Scene-Adaptive Trajectory-Controllable Human Image Animation with Camera Movement [51.22216867938034]
  We present a 3D motion animation framework that controls both human motion trajectories within a reconstructed 3D environment for video generation.<n>We design a viewpoint-adaptive latent fusion mechanism to inject point-cloud geometric priors through scene-visibility masking into the generative process.<n>Experiments on two standard human image animation benchmark datasets demonstrate remarkable improvements of our method over the state of the arts in related video generation metics.
  arXiv  Detail & Related papers  (2026-06-29T16:22:54Z)
• Sketch2Motion: Text-driven 2D Sketch to 3D Animation via Diffusion-guided Skeleton Optimization [0.6445605125467574]
  Animation of 2D hand-drawn sketches provides an effective medium for visual communication.<n>Recent approaches to converting 2D sketches to 3D animations have mainly focused on specific types of motion.<n>We propose Sketch2Motion, a diffusion-guided framework for skeleton-based motion synthesis.
  arXiv  Detail & Related papers  (2026-05-27T12:32:07Z)
• Animating the Uncaptured: Humanoid Mesh Animation with Video Diffusion Models [71.78723353724493]
  Animation of humanoid characters is essential in various graphics applications.<n>We propose an approach to synthesize 4D animated sequences of input static 3D humanoid meshes.
  arXiv  Detail & Related papers  (2025-03-20T10:00:22Z)
• Motion-2-to-3: Leveraging 2D Motion Data to Boost 3D Motion Generation [43.915871360698546]
  2D human videos offer a vast and accessible source of motion data, covering a wider range of styles and activities.<n>We introduce a novel framework that disentangles local joint motion from global movements, enabling efficient learning of local motion priors from 2D data.<n>Our method efficiently utilizes 2D data, supporting realistic 3D human motion generation and broadening the range of motion types it supports.
  arXiv  Detail & Related papers  (2024-12-17T17:34:52Z)
• Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics [79.4785166021062]
  We introduce Puppet-Master, an interactive video generator that captures the internal, part-level motion of objects.<n>We demonstrate that Puppet-Master learns to generate part-level motions, unlike other motion-conditioned video generators.<n>Puppet-Master generalizes well to out-of-domain real images, outperforming existing methods on real-world benchmarks.
  arXiv  Detail & Related papers  (2024-08-08T17:59:38Z)
• OmniMotionGPT: Animal Motion Generation with Limited Data [70.35662376853163]
  We introduce AnimalML3D, the first text-animal motion dataset with 1240 animation sequences spanning 36 different animal identities. We are able to generate animal motions with high diversity and fidelity, quantitatively and qualitatively outperforming the results of training human motion generation baselines on animal data.
  arXiv  Detail & Related papers  (2023-11-30T07:14:00Z)
• QS-Craft: Learning to Quantize, Scrabble and Craft for Conditional Human Motion Animation [66.97112599818507]
  This paper studies the task of conditional Human Motion Animation (cHMA) Given a source image and a driving video, the model should animate the new frame sequence. The key novelties come from the newly introduced three key steps: quantize, scrabble and craft.
  arXiv  Detail & Related papers  (2022-03-22T11:34:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.