TeMuDance: Contrastive Alignment-Based Textual Control for Music-Driven Dance Generation

• URL: http://arxiv.org/abs/2604.17005v1
• Date: Sat, 18 Apr 2026 14:24:57 GMT
• Title: TeMuDance: Contrastive Alignment-Based Textual Control for Music-Driven Dance Generation
• Abstract summary: TeMuDance is a framework that enables text-based control for music-conditioned dance generation.<n>It uses motion as a shared semantic anchor to align disjoint music-dance and text-motion datasets.<n>It achieves competitive dance quality while substantially improving text-conditioned control over existing methods.
• Score: 32.964897427586166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing music-driven dance generation approaches have achieved strong realism and effective audio-motion alignment. However, they generally lack semantic controllability, making it difficult to guide specific movements through natural language descriptions. This limitation primarily stems from the absence of large-scale datasets that jointly align music, text, and motion for supervised learning of text-conditioned control. To address this challenge, we propose TeMuDance, a framework that enables text-based control for music-conditioned dance generation without requiring any manually annotated music-text-motion triplet dataset. TeMuDance introduces a motion-centred bridging paradigm that leverages motion as a shared semantic anchor to align disjoint music-dance and text-motion datasets within a unified embedding space, enabling cross-modal retrieval of missing modalities for end-to-end training. A lightweight text control branch is then trained on top of a frozen music-to-dance diffusion backbone, preserving rhythmic fidelity while enabling fine-grained semantic guidance. To further suppress noise inherent in the retrieved supervision, we design a dual-stream fine-tuning strategy with confidence-based filtering. We also propose a novel task-aligned metric that quantifies whether textual prompts induce the intended kinematic attributes under music conditioning. Extensive experiments demonstrate that TeMuDance achieves competitive dance quality while substantially improving text-conditioned control over existing methods.

Related papers

• Music-to-Dance Generation via Atomic Movements [67.09789936557199]
  Music-driven dance generation aims to produce human motion that is rhythmically synchronized and semantically consistent with music.<n>We introduce a structure-aware framework that models choreography as a sequence of atomic movements-semantically interpretable motion events.<n>Our method produces dances with significantly improved structural coherence, rhythmic alignment, and perceptual naturalness compared to existing baselines.
  arXiv  Detail & Related papers  (2026-07-15T16:03:55Z)
• Text Dictates, Music Decorates: Energy-based Attention for Editable Dance Motion Generation [11.963078031832088]
  STREAM is a modality-decoupled diffusion transformer for Choreographic Motion Generation.<n>We introduce Motorica++, a dataset enriched with domain-specific dance vocabulary and frame-level semantic annotations.<n>STREAM achieves state-of-the-art alignment between motion and music while perfectly preserving choreographic semantics.
  arXiv  Detail & Related papers  (2026-06-22T00:01:05Z)
• MotionBeat: Motion-Aligned Music Representation via Embodied Contrastive Learning and Bar-Equivariant Contact-Aware Encoding [13.25040795516169]
  MotionBeat is a framework for motion-aligned music representation learning.<n>We show that MotionBeat outperforms state-of-the-art audio encoders in music-to-dance generation.
  arXiv  Detail & Related papers  (2025-10-15T07:44:32Z)
• MDD: A Dataset for Text-and-Music Conditioned Duet Dance Generation [16.657210427678198]
  We introduce Multimodal DuetDance (MDD), a diverse multimodal benchmark dataset designed for text-controlled and music-conditioned 3D duet dance motion generation.<n>Our dataset comprises 620 minutes of high-quality motion capture data performed by professional dancers, synchronized with music, and detailed with over 10K fine-grained natural language descriptions.<n>The annotations capture a rich movement vocabulary, detailing spatial relationships, body movements, and rhythm, making MDD the first dataset to seamlessly integrate human motions, music, and text for duet dance generation.
  arXiv  Detail & Related papers  (2025-08-23T05:56:37Z)
• DanceChat: Large Language Model-Guided Music-to-Dance Generation [8.455652926559427]
  Music-to-dance generation aims to synthesize human dance motion conditioned on musical input.<n>We introduce DanceChat, a Large Language Model (LLM)-guided music-to-dance generation approach.
  arXiv  Detail & Related papers  (2025-06-12T11:03:47Z)
• GCDance: Genre-Controlled Music-Driven 3D Full Body Dance Generation [30.028340528694432]
  GCDance is a framework for genre-specific 3D full-body dance generation conditioned on music and descriptive text.<n>We develop a text-based control mechanism that maps input prompts, explicit genre labels or free-form descriptive text, into genre-specific control signals.<n>To balance the objectives of extracting text-genre information and maintaining high-quality generation results, we propose a novel multi-task optimization strategy.
  arXiv  Detail & Related papers  (2025-02-25T15:53:18Z)
• Controllable Dance Generation with Style-Guided Motion Diffusion [49.35282418951445]
  Dance plays an important role as an artistic form and expression in human culture, yet the creation of dance remains a challenging task.<n>Most dance generation methods rely solely on music, seldom taking into consideration intrinsic attributes such as music style or genre.<n>In this work, we introduce Flexible Dance Generation with Style Description Prompts (DGSDP), a diffusion-based framework suitable for diversified tasks of dance generation.
  arXiv  Detail & Related papers  (2024-06-12T04:55:14Z)
• TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration [75.37311932218773]
  We propose a novel task for generating 3D dance movements that simultaneously incorporate both text and music modalities. Our approach can generate realistic and coherent dance movements conditioned on both text and music while maintaining comparable performance with the two single modalities.
  arXiv  Detail & Related papers  (2023-04-05T12:58:33Z)
• BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis [123.73677487809418]
  We introduce a new dataset aiming to challenge common assumptions in dance motion synthesis. We focus on breakdancing which features acrobatic moves and tangled postures. Our efforts produced the BRACE dataset, which contains over 3 hours and 30 minutes of densely annotated poses.
  arXiv  Detail & Related papers  (2022-07-20T18:03:54Z)
• Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory [92.81383016482813]
  We propose a novel music-to-dance framework, Bailando, for driving 3D characters to dance following a piece of music. We introduce an actor-critic Generative Pre-trained Transformer (GPT) that composes units to a fluent dance coherent to the music. Our proposed framework achieves state-of-the-art performance both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-03-24T13:06:43Z)
• Music-to-Dance Generation with Optimal Transport [48.92483627635586]
  We propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning to generate 3D dance choreographs from music. We introduce an optimal transport distance for evaluating the authenticity of the generated dance distribution and a Gromov-Wasserstein distance to measure the correspondence between the dance distribution and the input music.
  arXiv  Detail & Related papers  (2021-12-03T09:37:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.