TokenDance: Token-to-Token Music-to-Dance Generation with Bidirectional Mamba

• URL: http://arxiv.org/abs/2603.27314v1
• Date: Sat, 28 Mar 2026 15:38:14 GMT
• Title: TokenDance: Token-to-Token Music-to-Dance Generation with Bidirectional Mamba
• Abstract summary: Music-to-dance generation has broad applications in virtual reality, dance education, and digital character animation.<n>We present TokenDance, a two-stage music-to-dance generation framework that explicitly addresses this limitation through dual-modality tokenization and efficient token-level generation.<n> TokenDance achieves overall state-of-the-art (SOTA) performance in both generation quality and inference speed, highlighting its effectiveness and practical value for real-world music-to-dance applications.
• Score: 5.119197329627647
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Music-to-dance generation has broad applications in virtual reality, dance education, and digital character animation. However, the limited coverage of existing 3D dance datasets confines current models to a narrow subset of music styles and choreographic patterns, resulting in poor generalization to real-world music. Consequently, generated dances often become overly simplistic and repetitive, substantially degrading expressiveness and realism. To tackle this problem, we present TokenDance, a two-stage music-to-dance generation framework that explicitly addresses this limitation through dual-modality tokenization and efficient token-level generation. In the first stage, we discretize both dance and music using Finite Scalar Quantization, where dance motions are factorized into upper and lower-body components with kinematic-dynamic constraints, and music is decomposed into semantic and acoustic features with dedicated codebooks to capture choreography-specific structures. In the second stage, we introduce a Local-Global-Local token-to-token generator built on a Bidirectional Mamba backbone, enabling coherent motion synthesis, strong music-dance alignment, and efficient non-autoregressive inference. Extensive experiments demonstrate that TokenDance achieves overall state-of-the-art (SOTA) performance in both generation quality and inference speed, highlighting its effectiveness and practical value for real-world music-to-dance applications.

Related papers

• InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalization [54.47757841373109]
  This work aims to push the frontier of generalizable 3D dance generation by scaling up both data and model design.<n>We develop a fully automated pipeline that reconstructs high-fidelity 3D dance motions from monocular videos.<n>To eliminate the physical artifacts prevalent in existing reconstruction methods, we introduce a Foot Restoration Diffusion Model (FRDM) guided by foot-contact and geometric constraints.<n>To enhance robustness under unfamiliar music conditions, we integrate a retrieval-augmented generation (RAG) module that injects reference dance as a prompt.
  arXiv  Detail & Related papers  (2026-03-10T10:10:34Z)
• DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling [70.79846001735547]
  We present DuetGen, a framework for generating interactive two-person dances from music.<n>Inspired by the recent advances in motion synthesis, we propose a two-stage solution.<n>We represent both dancers' motions as a unified whole to learn the necessary motion tokens.
  arXiv  Detail & Related papers  (2025-06-23T14:22:50Z)
• X-Dancer: Expressive Music to Human Dance Video Generation [26.544761204917336]
  X-Dancer is a novel zero-shot music-driven image animation pipeline.<n>It creates diverse and long-range lifelike human dance videos from a single static image.
  arXiv  Detail & Related papers  (2025-02-24T18:47:54Z)
• Controllable Dance Generation with Style-Guided Motion Diffusion [49.35282418951445]
  Dance plays an important role as an artistic form and expression in human culture, yet the creation of dance remains a challenging task.<n>Most dance generation methods rely solely on music, seldom taking into consideration intrinsic attributes such as music style or genre.<n>In this work, we introduce Flexible Dance Generation with Style Description Prompts (DGSDP), a diffusion-based framework suitable for diversified tasks of dance generation.
  arXiv  Detail & Related papers  (2024-06-12T04:55:14Z)
• Bidirectional Autoregressive Diffusion Model for Dance Generation [26.449135437337034]
  We propose a Bidirectional Autoregressive Diffusion Model (BADM) for music-to-dance generation. A bidirectional encoder is built to enforce that the generated dance is harmonious in both the forward and backward directions. To make the generated dance motion smoother, a local information decoder is built for local motion enhancement.
  arXiv  Detail & Related papers  (2024-02-06T19:42:18Z)
• TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration [75.37311932218773]
  We propose a novel task for generating 3D dance movements that simultaneously incorporate both text and music modalities. Our approach can generate realistic and coherent dance movements conditioned on both text and music while maintaining comparable performance with the two single modalities.
  arXiv  Detail & Related papers  (2023-04-05T12:58:33Z)
• Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory [92.81383016482813]
  We propose a novel music-to-dance framework, Bailando, for driving 3D characters to dance following a piece of music. We introduce an actor-critic Generative Pre-trained Transformer (GPT) that composes units to a fluent dance coherent to the music. Our proposed framework achieves state-of-the-art performance both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-03-24T13:06:43Z)
• Music-to-Dance Generation with Optimal Transport [48.92483627635586]
  We propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning to generate 3D dance choreographs from music. We introduce an optimal transport distance for evaluating the authenticity of the generated dance distribution and a Gromov-Wasserstein distance to measure the correspondence between the dance distribution and the input music.
  arXiv  Detail & Related papers  (2021-12-03T09:37:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.