How Animals Dance (When You're Not Looking)

• URL: http://arxiv.org/abs/2505.23738v1
• Date: Thu, 29 May 2025 17:58:02 GMT
• Title: How Animals Dance (When You're Not Looking)
• Authors: Xiaojuan Wang, Aleksander Holynski, Brian Curless, Ira Kemelmacher, Steve Seitz,
• Abstract summary: We present a framework for generating music-aware, aware animal dance videos.<n>With as few as six input diffusions, our method can produce up to 30 second dance videos.
• Score: 50.76342313977405
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present a keyframe-based framework for generating music-synchronized, choreography aware animal dance videos. Starting from a few keyframes representing distinct animal poses -- generated via text-to-image prompting or GPT-4o -- we formulate dance synthesis as a graph optimization problem: find the optimal keyframe structure that satisfies a specified choreography pattern of beats, which can be automatically estimated from a reference dance video. We also introduce an approach for mirrored pose image generation, essential for capturing symmetry in dance. In-between frames are synthesized using an video diffusion model. With as few as six input keyframes, our method can produce up to 30 second dance videos across a wide range of animals and music tracks.

Related papers

• ChoreoMuse: Robust Music-to-Dance Video Generation with Style Transfer and Beat-Adherent Motion [10.21851621470535]
  We introduce ChoreoMuse, a diffusion-based framework that uses SMPL format parameters and their variation version as intermediaries between music and video generation.<n>ChoreoMuse supports style-controllable, high-fidelity dance video generation across diverse musical genres and individual dancer characteristics.<n>Our method employs a novel music encoder MotionTune to capture motion cues from audio, ensuring that the generated choreography closely follows the beat and expressive qualities of the input music.
  arXiv  Detail & Related papers  (2025-07-26T07:17:50Z)
• Let Your Video Listen to Your Music! [62.27731415767459]
  We propose a novel framework, MVAA, that automatically edits video to align with the rhythm of a given music track.<n>We modularize the task into a two-step process in our MVAA: aligning motion with audio beats, followed by rhythm-aware video editing.<n>This hybrid approach enables adaptation within 10 minutes with one on a single NVIDIA 4090 GPU using CogVideoX-5b-I2V as the backbone.
  arXiv  Detail & Related papers  (2025-06-23T17:52:16Z)
• X-Dancer: Expressive Music to Human Dance Video Generation [26.544761204917336]
  X-Dancer is a novel zero-shot music-driven image animation pipeline.<n>It creates diverse and long-range lifelike human dance videos from a single static image.
  arXiv  Detail & Related papers  (2025-02-24T18:47:54Z)
• Every Image Listens, Every Image Dances: Music-Driven Image Animation [8.085267959520843]
  MuseDance is an end-to-end model that animates reference images using both music and text inputs.<n>Unlike existing approaches, MuseDance eliminates the need for complex motion guidance inputs, such as pose or depth sequences.<n>We present a new multimodal dataset comprising 2,904 dance videos with corresponding background music and text descriptions.
  arXiv  Detail & Related papers  (2025-01-30T23:38:51Z)
• DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis [49.614150163184064]
  Dance camera movements involve both continuous sequences of variable lengths and sudden changes to simulate the switching of multiple cameras. We propose to integrate cinematography knowledge by formulating this task as a three-stage process: animator detection, synthesis, and tween function prediction. Following this formulation, we design a novel end-to-end dance camera framework textbfDanceCamAnimator, which imitates human animation procedures and shows powerful-based controllability with variable lengths.
  arXiv  Detail & Related papers  (2024-09-23T11:20:44Z)
• Dance Any Beat: Blending Beats with Visuals in Dance Video Generation [12.018432669719742]
  We introduce a novel task: generating dance videos directly from images of individuals guided by music.<n>Our solution, the Dance Any Beat Diffusion model (DabFusion), utilizes a reference image and a music piece to generate dance videos.<n>We evaluate DabFusion's performance using the AIST++ dataset, focusing on video quality, audio-video synchronization, and motion-music alignment.
  arXiv  Detail & Related papers  (2024-05-15T11:33:07Z)
• BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis [123.73677487809418]
  We introduce a new dataset aiming to challenge common assumptions in dance motion synthesis. We focus on breakdancing which features acrobatic moves and tangled postures. Our efforts produced the BRACE dataset, which contains over 3 hours and 30 minutes of densely annotated poses.
  arXiv  Detail & Related papers  (2022-07-20T18:03:54Z)
• MetaDance: Few-shot Dancing Video Retargeting via Temporal-aware Meta-learning [51.78302763617991]
  Dancing video aims to synthesize a video that transfers the dance movements from a source video to a target person. Previous work need collect a several-minute-long video of a target person with thousands of frames to train a personalized model. Recent work tackled few-shot dancing video, which learns to synthesize videos of unseen persons by leveraging a few frames of them.
  arXiv  Detail & Related papers  (2022-01-13T09:34:20Z)
• Learning to Generate Diverse Dance Motions with Transformer [67.43270523386185]
  We introduce a complete system for dance motion synthesis. A massive dance motion data set is created from YouTube videos. A novel two-stream motion transformer generative model can generate motion sequences with high flexibility.
  arXiv  Detail & Related papers  (2020-08-18T22:29:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.