Dance Generation by Sound Symbolic Words

• URL: http://arxiv.org/abs/2306.03646v1
• Date: Tue, 6 Jun 2023 13:00:47 GMT
• Title: Dance Generation by Sound Symbolic Words
• Abstract summary: This study introduces a novel approach to generate dance motions using onomatopoeia as input. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions. We present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey.
• Score: 5.004501184476518
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study introduces a novel approach to generate dance motions using onomatopoeia as input, with the aim of enhancing creativity and diversity in dance generation. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions without constraints on expression and without need for specialized knowledge. We adapt the AI Choreographer framework and employ the Sakamoto system, a feature extraction method for onomatopoeia focusing on phonemes and syllables. Additionally, we present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey. Our results demonstrate that the proposed method enables more intuitive dance generation and can create dance motions using sound-symbolic words from a variety of languages, including those without onomatopoeia. This highlights the potential for diverse dance creation across different languages and cultures, accessible to a wider audience. Qualitative samples from our model can be found at: https://sites.google.com/view/onomatopoeia-dance/home/.

Related papers

• Not Like Transformers: Drop the Beat Representation for Dance Generation with Mamba-Based Diffusion Model [8.542693293312935]
  We propose emphMambaDance, a new dance generation approach that leverages a Mamba-based diffusion model.<n>Mamba, well-suited to handling long and autoregressive sequences, is integrated into our two-stage diffusion architecture, substituting off-the-shelf Transformer.<n>We show that our proposed method effectively generates plausible dance movements while reflecting essential characteristics, consistently from short to long dances, compared to the previous methods.
  arXiv  Detail & Related papers  (2026-03-09T06:59:03Z)
• Think2Sing: Orchestrating Structured Motion Subtitles for Singing-Driven 3D Head Animation [69.50178144839275]
  Singing involves richer emotional nuance, dynamic prosody, and lyric-based semantics.<n>Existing speech-driven approaches often produce oversimplified, emotionally flat, and semantically inconsistent results.<n>Think2Sing generates semantically coherent and temporally consistent 3D head animations conditioned on both lyrics and acoustics.
  arXiv  Detail & Related papers  (2025-09-02T12:59:27Z)
• DanceChat: Large Language Model-Guided Music-to-Dance Generation [8.455652926559427]
  Music-to-dance generation aims to synthesize human dance motion conditioned on musical input.<n>We introduce DanceChat, a Large Language Model (LLM)-guided music-to-dance generation approach.
  arXiv  Detail & Related papers  (2025-06-12T11:03:47Z)
• Align Your Rhythm: Generating Highly Aligned Dance Poses with Gating-Enhanced Rhythm-Aware Feature Representation [22.729568599120846]
  We propose Danceba, a novel framework that leverages gating mechanism to enhance rhythm-aware feature representation. Phase-Based Rhythm Extraction (PRE) to precisely extract rhythmic information from musical phase data. Temporal-Gated Causal Attention (TGCA) to focus on global rhythmic features. Parallel Mamba Motion Modeling (PMMM) architecture to separately model upper and lower body motions.
  arXiv  Detail & Related papers  (2025-03-21T17:42:50Z)
• GCDance: Genre-Controlled Music-Driven 3D Full Body Dance Generation [30.028340528694432]
  GCDance is a framework for genre-specific 3D full-body dance generation conditioned on music and descriptive text.<n>We develop a text-based control mechanism that maps input prompts, explicit genre labels or free-form descriptive text, into genre-specific control signals.<n>To balance the objectives of extracting text-genre information and maintaining high-quality generation results, we propose a novel multi-task optimization strategy.
  arXiv  Detail & Related papers  (2025-02-25T15:53:18Z)
• Flexible Music-Conditioned Dance Generation with Style Description Prompts [41.04549275897979]
  We introduce Flexible Dance Generation with Style Description Prompts (DGSDP), a diffusion-based framework suitable for diversified tasks of dance generation. The core component of this framework is Music-Conditioned Style-Aware Diffusion (MCSAD), which comprises a Transformer-based network and a music Style Modulation module. The proposed framework successfully generates realistic dance sequences that are accurately aligned with music for a variety of tasks such as long-term generation, dance in-betweening, dance inpainting, and etc.
  arXiv  Detail & Related papers  (2024-06-12T04:55:14Z)
• LM2D: Lyrics- and Music-Driven Dance Synthesis [28.884929875333846]
  LM2D is designed to create dance conditioned on both music and lyrics in one diffusion generation step. We introduce the first 3D dance-motion dataset that encompasses both music and lyrics, obtained with pose estimation technologies. The results demonstrate LM2D is able to produce realistic and diverse dance matching both lyrics and music.
  arXiv  Detail & Related papers  (2024-03-14T13:59:04Z)
• Bidirectional Autoregressive Diffusion Model for Dance Generation [26.449135437337034]
  We propose a Bidirectional Autoregressive Diffusion Model (BADM) for music-to-dance generation. A bidirectional encoder is built to enforce that the generated dance is harmonious in both the forward and backward directions. To make the generated dance motion smoother, a local information decoder is built for local motion enhancement.
  arXiv  Detail & Related papers  (2024-02-06T19:42:18Z)
• TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration [75.37311932218773]
  We propose a novel task for generating 3D dance movements that simultaneously incorporate both text and music modalities. Our approach can generate realistic and coherent dance movements conditioned on both text and music while maintaining comparable performance with the two single modalities.
  arXiv  Detail & Related papers  (2023-04-05T12:58:33Z)
• BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis [123.73677487809418]
  We introduce a new dataset aiming to challenge common assumptions in dance motion synthesis. We focus on breakdancing which features acrobatic moves and tangled postures. Our efforts produced the BRACE dataset, which contains over 3 hours and 30 minutes of densely annotated poses.
  arXiv  Detail & Related papers  (2022-07-20T18:03:54Z)
• Quantized GAN for Complex Music Generation from Dance Videos [48.196705493763986]
  We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates musical samples conditioned on dance videos. Our proposed framework takes dance video frames and human body motion as input, and learns to generate music samples that plausibly accompany the corresponding input.
  arXiv  Detail & Related papers  (2022-04-01T17:53:39Z)
• Music-to-Dance Generation with Optimal Transport [48.92483627635586]
  We propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning to generate 3D dance choreographs from music. We introduce an optimal transport distance for evaluating the authenticity of the generated dance distribution and a Gromov-Wasserstein distance to measure the correspondence between the dance distribution and the input music.
  arXiv  Detail & Related papers  (2021-12-03T09:37:26Z)
• Learning to Generate Diverse Dance Motions with Transformer [67.43270523386185]
  We introduce a complete system for dance motion synthesis. A massive dance motion data set is created from YouTube videos. A novel two-stream motion transformer generative model can generate motion sequences with high flexibility.
  arXiv  Detail & Related papers  (2020-08-18T22:29:40Z)
• Feel The Music: Automatically Generating A Dance For An Input Song [58.653867648572]
  We present a general computational approach that enables a machine to generate a dance for any input music. We encode intuitive, flexibles for what a 'good' dance is: the structure of the dance should align with the structure of the music.
  arXiv  Detail & Related papers  (2020-06-21T20:29:50Z)
• Bharatanatyam Dance Transcription using Multimedia Ontology and Machine Learning [0.0]
  We develop a system to generate a parseable representation of a dance performance. The system will help to preserve intangible heritage, annotate performances for better tutoring, and synthesize dance performances.
  arXiv  Detail & Related papers  (2020-04-24T21:22:20Z)
• Music2Dance: DanceNet for Music-driven Dance Generation [11.73506542921528]
  We propose a novel autoregressive generative model, DanceNet, to take the style, rhythm and melody of music as the control signals. We capture several synchronized music-dance pairs by professional dancers, and build a high-quality music-dance pair dataset.
  arXiv  Detail & Related papers  (2020-02-02T17:18:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.