EDGE: Editable Dance Generation From Music
- URL: http://arxiv.org/abs/2211.10658v1
- Date: Sat, 19 Nov 2022 10:41:38 GMT
- Title: EDGE: Editable Dance Generation From Music
- Authors: Jonathan Tseng, Rodrigo Castellon, C. Karen Liu
- Abstract summary: We introduce Editable Dance GEneration (EDGE), a state-of-the-art method for editable dance generation.
It is capable of creating realistic, physically-plausible dances while remaining faithful to the input music.
- Score: 15.658163494375533
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Dance is an important human art form, but creating new dances can be
difficult and time-consuming. In this work, we introduce Editable Dance
GEneration (EDGE), a state-of-the-art method for editable dance generation that
is capable of creating realistic, physically-plausible dances while remaining
faithful to the input music. EDGE uses a transformer-based diffusion model
paired with Jukebox, a strong music feature extractor, and confers powerful
editing capabilities well-suited to dance, including joint-wise conditioning,
and in-betweening. We introduce a new metric for physical plausibility, and
evaluate dance quality generated by our method extensively through (1) multiple
quantitative metrics on physical plausibility, beat alignment, and diversity
benchmarks, and more importantly, (2) a large-scale user study, demonstrating a
significant improvement over previous state-of-the-art methods. Qualitative
samples from our model can be found at our website.
Related papers
- DanceChat: Large Language Model-Guided Music-to-Dance Generation [8.455652926559427]
Music-to-dance generation aims to synthesize human dance motion conditioned on musical input.<n>We introduce DanceChat, a Large Language Model (LLM)-guided music-to-dance generation approach.
arXiv Detail & Related papers (2025-06-12T11:03:47Z) - GCDance: Genre-Controlled 3D Full Body Dance Generation Driven By Music [22.352036716156967]
GCDance is a classifier-free diffusion framework for generating genre-specific dance motions conditioned on both music and textual prompts.
Our approach extracts music features by combining high-level pre-trained music foundation model features with hand-crafted features for multi-granularity feature fusion.
arXiv Detail & Related papers (2025-02-25T15:53:18Z) - Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns [48.54956784928394]
Lodge++ is a choreography framework to generate high-quality, ultra-long, and vivid dances given the music and desired genre.
To handle the challenges in computational efficiency, Lodge++ adopts a two-stage strategy to produce dances from coarse to fine.
Lodge++ is validated by extensive experiments, which show that our method can rapidly generate ultra-long dances suitable for various dance genres.
arXiv Detail & Related papers (2024-10-27T09:32:35Z) - Flexible Music-Conditioned Dance Generation with Style Description Prompts [41.04549275897979]
We introduce Flexible Dance Generation with Style Description Prompts (DGSDP), a diffusion-based framework suitable for diversified tasks of dance generation.
The core component of this framework is Music-Conditioned Style-Aware Diffusion (MCSAD), which comprises a Transformer-based network and a music Style Modulation module.
The proposed framework successfully generates realistic dance sequences that are accurately aligned with music for a variety of tasks such as long-term generation, dance in-betweening, dance inpainting, and etc.
arXiv Detail & Related papers (2024-06-12T04:55:14Z) - Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment [87.20240797625648]
We introduce a novel task within the field of 3D dance generation, termed dance accompaniment.
It requires the generation of responsive movements from a dance partner, the "follower", synchronized with the lead dancer's movements and the underlying musical rhythm.
We propose a GPT-based model, Duolando, which autoregressively predicts the subsequent tokenized motion conditioned on the coordinated information of the music, the leader's and the follower's movements.
arXiv Detail & Related papers (2024-03-27T17:57:02Z) - DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation [89.50310360658791]
We present a novel cascaded motion diffusion model, DiffDance, designed for high-resolution, long-form dance generation.
This model comprises a music-to-dance diffusion model and a sequence super-resolution diffusion model.
We demonstrate that DiffDance is capable of generating realistic dance sequences that align effectively with the input music.
arXiv Detail & Related papers (2023-08-05T16:18:57Z) - FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance
Generation [33.9261932800456]
FineDance is the largest music-dance paired dataset with the most dance genres.
To address monotonous and unnatural hand movements existing in previous methods, we propose a full-body dance generation network.
To further enhance the genre-matching and long-term stability of generated dances, we propose a Genre&Coherent aware Retrieval Module.
arXiv Detail & Related papers (2022-12-07T16:10:08Z) - Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic
Memory [92.81383016482813]
We propose a novel music-to-dance framework, Bailando, for driving 3D characters to dance following a piece of music.
We introduce an actor-critic Generative Pre-trained Transformer (GPT) that composes units to a fluent dance coherent to the music.
Our proposed framework achieves state-of-the-art performance both qualitatively and quantitatively.
arXiv Detail & Related papers (2022-03-24T13:06:43Z) - Music-to-Dance Generation with Optimal Transport [48.92483627635586]
We propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning to generate 3D dance choreographs from music.
We introduce an optimal transport distance for evaluating the authenticity of the generated dance distribution and a Gromov-Wasserstein distance to measure the correspondence between the dance distribution and the input music.
arXiv Detail & Related papers (2021-12-03T09:37:26Z) - Learning to Generate Diverse Dance Motions with Transformer [67.43270523386185]
We introduce a complete system for dance motion synthesis.
A massive dance motion data set is created from YouTube videos.
A novel two-stream motion transformer generative model can generate motion sequences with high flexibility.
arXiv Detail & Related papers (2020-08-18T22:29:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.