DanceEditor: Towards Iterative Editable Music-driven Dance Generation with Open-Vocabulary Descriptions

• URL: http://arxiv.org/abs/2508.17342v1
• Date: Sun, 24 Aug 2025 12:53:09 GMT
• Title: DanceEditor: Towards Iterative Editable Music-driven Dance Generation with Open-Vocabulary Descriptions
• Abstract summary: We propose a novel framework for iterative and editable dance generation coherently aligned with given music signals.<n>Our framework is built upon a prediction-then-editing paradigm unifying multi-modal conditions.<n>Our method outperforms the state-of-the-art models on our newly collected DanceRemix dataset.
• Score: 22.816939666772303
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating coherent and diverse human dances from music signals has gained tremendous progress in animating virtual avatars. While existing methods support direct dance synthesis, they fail to recognize that enabling users to edit dance movements is far more practical in real-world choreography scenarios. Moreover, the lack of high-quality dance datasets incorporating iterative editing also limits addressing this challenge. To achieve this goal, we first construct DanceRemix, a large-scale multi-turn editable dance dataset comprising the prompt featuring over 25.3M dance frames and 84.5K pairs. In addition, we propose a novel framework for iterative and editable dance generation coherently aligned with given music signals, namely DanceEditor. Considering the dance motion should be both musical rhythmic and enable iterative editing by user descriptions, our framework is built upon a prediction-then-editing paradigm unifying multi-modal conditions. At the initial prediction stage, our framework improves the authority of generated results by directly modeling dance movements from tailored, aligned music. Moreover, at the subsequent iterative editing stages, we incorporate text descriptions as conditioning information to draw the editable results through a specifically designed Cross-modality Editing Module (CEM). Specifically, CEM adaptively integrates the initial prediction with music and text prompts as temporal motion cues to guide the synthesized sequences. Thereby, the results display music harmonics while preserving fine-grained semantic alignment with text descriptions. Extensive experiments demonstrate that our method outperforms the state-of-the-art models on our newly collected DanceRemix dataset. Code is available at https://lzvsdy.github.io/DanceEditor/.

Related papers

• Music-to-Dance Generation via Atomic Movements [67.09789936557199]
  Music-driven dance generation aims to produce human motion that is rhythmically synchronized and semantically consistent with music.<n>We introduce a structure-aware framework that models choreography as a sequence of atomic movements-semantically interpretable motion events.<n>Our method produces dances with significantly improved structural coherence, rhythmic alignment, and perceptual naturalness compared to existing baselines.
  arXiv  Detail & Related papers  (2026-07-15T16:03:55Z)
• TeMuDance: Contrastive Alignment-Based Textual Control for Music-Driven Dance Generation [32.964897427586166]
  TeMuDance is a framework that enables text-based control for music-conditioned dance generation.<n>It uses motion as a shared semantic anchor to align disjoint music-dance and text-motion datasets.<n>It achieves competitive dance quality while substantially improving text-conditioned control over existing methods.
  arXiv  Detail & Related papers  (2026-04-18T14:24:57Z)
• Not Like Transformers: Drop the Beat Representation for Dance Generation with Mamba-Based Diffusion Model [8.542693293312935]
  We propose emphMambaDance, a new dance generation approach that leverages a Mamba-based diffusion model.<n>Mamba, well-suited to handling long and autoregressive sequences, is integrated into our two-stage diffusion architecture, substituting off-the-shelf Transformer.<n>We show that our proposed method effectively generates plausible dance movements while reflecting essential characteristics, consistently from short to long dances, compared to the previous methods.
  arXiv  Detail & Related papers  (2026-03-09T06:59:03Z)
• DanceChat: Large Language Model-Guided Music-to-Dance Generation [8.455652926559427]
  Music-to-dance generation aims to synthesize human dance motion conditioned on musical input.<n>We introduce DanceChat, a Large Language Model (LLM)-guided music-to-dance generation approach.
  arXiv  Detail & Related papers  (2025-06-12T11:03:47Z)
• X-Dancer: Expressive Music to Human Dance Video Generation [26.544761204917336]
  X-Dancer is a novel zero-shot music-driven image animation pipeline.<n>It creates diverse and long-range lifelike human dance videos from a single static image.
  arXiv  Detail & Related papers  (2025-02-24T18:47:54Z)
• DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis [49.614150163184064]
  Dance camera movements involve both continuous sequences of variable lengths and sudden changes to simulate the switching of multiple cameras. We propose to integrate cinematography knowledge by formulating this task as a three-stage process: animator detection, synthesis, and tween function prediction. Following this formulation, we design a novel end-to-end dance camera framework textbfDanceCamAnimator, which imitates human animation procedures and shows powerful-based controllability with variable lengths.
  arXiv  Detail & Related papers  (2024-09-23T11:20:44Z)
• Controllable Dance Generation with Style-Guided Motion Diffusion [49.35282418951445]
  Dance plays an important role as an artistic form and expression in human culture, yet the creation of dance remains a challenging task.<n>Most dance generation methods rely solely on music, seldom taking into consideration intrinsic attributes such as music style or genre.<n>In this work, we introduce Flexible Dance Generation with Style Description Prompts (DGSDP), a diffusion-based framework suitable for diversified tasks of dance generation.
  arXiv  Detail & Related papers  (2024-06-12T04:55:14Z)
• Dance Any Beat: Blending Beats with Visuals in Dance Video Generation [12.018432669719742]
  We introduce a novel task: generating dance videos directly from images of individuals guided by music.<n>Our solution, the Dance Any Beat Diffusion model (DabFusion), utilizes a reference image and a music piece to generate dance videos.<n>We evaluate DabFusion's performance using the AIST++ dataset, focusing on video quality, audio-video synchronization, and motion-music alignment.
  arXiv  Detail & Related papers  (2024-05-15T11:33:07Z)
• TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration [75.37311932218773]
  We propose a novel task for generating 3D dance movements that simultaneously incorporate both text and music modalities. Our approach can generate realistic and coherent dance movements conditioned on both text and music while maintaining comparable performance with the two single modalities.
  arXiv  Detail & Related papers  (2023-04-05T12:58:33Z)
• BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis [123.73677487809418]
  We introduce a new dataset aiming to challenge common assumptions in dance motion synthesis. We focus on breakdancing which features acrobatic moves and tangled postures. Our efforts produced the BRACE dataset, which contains over 3 hours and 30 minutes of densely annotated poses.
  arXiv  Detail & Related papers  (2022-07-20T18:03:54Z)
• Quantized GAN for Complex Music Generation from Dance Videos [48.196705493763986]
  We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates musical samples conditioned on dance videos. Our proposed framework takes dance video frames and human body motion as input, and learns to generate music samples that plausibly accompany the corresponding input.
  arXiv  Detail & Related papers  (2022-04-01T17:53:39Z)
• Music-to-Dance Generation with Optimal Transport [48.92483627635586]
  We propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning to generate 3D dance choreographs from music. We introduce an optimal transport distance for evaluating the authenticity of the generated dance distribution and a Gromov-Wasserstein distance to measure the correspondence between the dance distribution and the input music.
  arXiv  Detail & Related papers  (2021-12-03T09:37:26Z)
• DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformer [23.51701359698245]
  In this paper, we reformulate it by a two-stage process, ie, a key pose generation and then an in-between parametric motion curve prediction. We propose a large-scale music conditioned 3D dance dataset, called PhantomDance, that is accurately labeled by experienced animators. Experiments demonstrate that the proposed method, even trained by existing datasets, can generate fluent, performative, and music-matched 3D dances.
  arXiv  Detail & Related papers  (2021-03-18T12:17:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.