InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalization

• URL: http://arxiv.org/abs/2603.13375v1
• Date: Tue, 10 Mar 2026 10:10:34 GMT
• Title: InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalization
• Abstract summary: This work aims to push the frontier of generalizable 3D dance generation by scaling up both data and model design.<n>We develop a fully automated pipeline that reconstructs high-fidelity 3D dance motions from monocular videos.<n>To eliminate the physical artifacts prevalent in existing reconstruction methods, we introduce a Foot Restoration Diffusion Model (FRDM) guided by foot-contact and geometric constraints.<n>To enhance robustness under unfamiliar music conditions, we integrate a retrieval-augmented generation (RAG) module that injects reference dance as a prompt.
• Score: 54.47757841373109
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although existing 3D dance generation methods perform well in controlled scenarios, they often struggle to generalize in the wild. When conditioned on unseen music, existing methods often produce unstructured or physically implausible dance, largely due to limited music-to-dance data and restricted model capacity. This work aims to push the frontier of generalizable 3D dance generation by scaling up both data and model design. (1) On the data side, we develop a fully automated pipeline that reconstructs high-fidelity 3D dance motions from monocular videos. To eliminate the physical artifacts prevalent in existing reconstruction methods, we introduce a Foot Restoration Diffusion Model (FRDM) guided by foot-contact and geometric constraints that enforce physical plausibility while preserving kinematic smoothness and expressiveness, resulting in a diverse, high-quality multimodal 3D dance dataset totaling 100.69 hours. (2) On model design, we propose Choreographic LLaMA (ChoreoLLaMA), a scalable LLaMA-based architecture. To enhance robustness under unfamiliar music conditions, we integrate a retrieval-augmented generation (RAG) module that injects reference dance as a prompt. Additionally, we design a slow/fast-cadence Mixture-of-Experts (MoE) module that enables ChoreoLLaMA to smoothly adapt motion rhythms across varying music tempos. Extensive experiments across diverse dance genres show that our approach surpasses existing methods in both qualitative and quantitative evaluations, marking a step toward scalable, real-world 3D dance generation. Code, models, and data will be released.

Related papers

• Listen to Rhythm, Choose Movements: Autoregressive Multimodal Dance Generation via Diffusion and Mamba with Decoupled Dance Dataset [8.721362823189077]
  Listen to Rhythm, Choose Movements (LRCM) is a multimodal-guided diffusion framework supporting both diverse input modalities and autoregressive dance motion generation.<n>We will release the full dataset, and pretrained models publicly upon acceptance.
  arXiv  Detail & Related papers  (2026-01-06T14:59:22Z)
• UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework [54.337290937468175]
  We propose UniMo, an autoregressive model for joint modeling of 2D human videos and 3D human motions within a unified framework.<n>We show that our method simultaneously generates corresponding videos and motions while performing accurate motion capture.
  arXiv  Detail & Related papers  (2025-12-03T16:03:18Z)
• FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generation [9.587067781689331]
  Music-to-dance generation aims to translate auditory signals into expressive human motion, with broad applications in virtual reality, choreography, and digital entertainment.<n>We propose FlowerDance, which not only generates refined motion with physical plausibility and artistic expressiveness, but also achieves significant generation efficiency on inference speed and memory utilization.
  arXiv  Detail & Related papers  (2025-11-26T03:53:10Z)
• PAMD: Plausibility-Aware Motion Diffusion Model for Long Dance Generation [51.2555550979386]
  Plausibility-Aware Motion Diffusion (PAMD) is a framework for generating dances that are both musically aligned and physically realistic.<n>To provide more effective guidance during generation, we incorporate Prior Motion Guidance (PMG)<n>Experiments show that PAMD significantly improves musical alignment and enhances the physical plausibility of generated motions.
  arXiv  Detail & Related papers  (2025-05-26T14:44:09Z)
• Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns [48.54956784928394]
  Lodge++ is a choreography framework to generate high-quality, ultra-long, and vivid dances given the music and desired genre. To handle the challenges in computational efficiency, Lodge++ adopts a two-stage strategy to produce dances from coarse to fine. Lodge++ is validated by extensive experiments, which show that our method can rapidly generate ultra-long dances suitable for various dance genres.
  arXiv  Detail & Related papers  (2024-10-27T09:32:35Z)
• LongDanceDiff: Long-term Dance Generation with Conditional Diffusion Model [3.036230795326545]
  LongDanceDiff is a conditional diffusion model for sequence-to-sequence long-term dance generation. It addresses the challenges of temporal coherency and spatial constraint. We also address common visual quality issues in dance generation, such as foot sliding and unsmooth motion.
  arXiv  Detail & Related papers  (2023-08-23T06:37:41Z)
• DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation [89.50310360658791]
  We present a novel cascaded motion diffusion model, DiffDance, designed for high-resolution, long-form dance generation. This model comprises a music-to-dance diffusion model and a sequence super-resolution diffusion model. We demonstrate that DiffDance is capable of generating realistic dance sequences that align effectively with the input music.
  arXiv  Detail & Related papers  (2023-08-05T16:18:57Z)
• TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration [75.37311932218773]
  We propose a novel task for generating 3D dance movements that simultaneously incorporate both text and music modalities. Our approach can generate realistic and coherent dance movements conditioned on both text and music while maintaining comparable performance with the two single modalities.
  arXiv  Detail & Related papers  (2023-04-05T12:58:33Z)
• FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation [33.9261932800456]
  FineDance is the largest music-dance paired dataset with the most dance genres. To address monotonous and unnatural hand movements existing in previous methods, we propose a full-body dance generation network. To further enhance the genre-matching and long-term stability of generated dances, we propose a Genre&Coherent aware Retrieval Module.
  arXiv  Detail & Related papers  (2022-12-07T16:10:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.