Global Position Aware Group Choreography using Large Language Model

• URL: http://arxiv.org/abs/2503.09645v1
• Date: Wed, 12 Mar 2025 07:25:32 GMT
• Title: Global Position Aware Group Choreography using Large Language Model
• Authors: Haozhou Pang, Tianwei Ding, Lanshan He, Qi Gan,
• Abstract summary: We present a group choreography framework that leverages recent advancements in Large Language Models (LLM)<n>Our framework consists of a tokenizer that transforms continuous features into discrete tokens, and an LLM that is fine-tuned to predict motion tokens.<n>Our framework can generate realistic and diverse group dances while maintaining strong music correlation and dancer-wise consistency.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dance serves as a profound and universal expression of human culture, conveying emotions and stories through movements synchronized with music. Although some current works have achieved satisfactory results in the task of single-person dance generation, the field of multi-person dance generation remains relatively novel. In this work, we present a group choreography framework that leverages recent advancements in Large Language Models (LLM) by modeling the group dance generation problem as a sequence-to-sequence translation task. Our framework consists of a tokenizer that transforms continuous features into discrete tokens, and an LLM that is fine-tuned to predict motion tokens given the audio tokens. We show that by proper tokenization of input modalities and careful design of the LLM training strategies, our framework can generate realistic and diverse group dances while maintaining strong music correlation and dancer-wise consistency. Extensive experiments and evaluations demonstrate that our framework achieves state-of-the-art performance.

Related papers

• GCDance: Genre-Controlled 3D Full Body Dance Generation Driven By Music [22.352036716156967]
  GCDance is a classifier-free diffusion framework for generating genre-specific dance motions conditioned on both music and textual prompts.<n>Our approach extracts music features by combining high-level pre-trained music foundation model features with hand-crafted features for multi-granularity feature fusion.
  arXiv  Detail & Related papers  (2025-02-25T15:53:18Z)
• Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns [48.54956784928394]
  Lodge++ is a choreography framework to generate high-quality, ultra-long, and vivid dances given the music and desired genre. To handle the challenges in computational efficiency, Lodge++ adopts a two-stage strategy to produce dances from coarse to fine. Lodge++ is validated by extensive experiments, which show that our method can rapidly generate ultra-long dances suitable for various dance genres.
  arXiv  Detail & Related papers  (2024-10-27T09:32:35Z)
• Scalable Group Choreography via Variational Phase Manifold Learning [8.504657927912076]
  We propose a phase-based variational generative model for group dance generation on learning a generative manifold. Our method achieves high-fidelity group dance motion and enables the generation with an unlimited number of dancers.
  arXiv  Detail & Related papers  (2024-07-26T16:02:37Z)
• Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment [87.20240797625648]
  We introduce a novel task within the field of 3D dance generation, termed dance accompaniment. It requires the generation of responsive movements from a dance partner, the "follower", synchronized with the lead dancer's movements and the underlying musical rhythm. We propose a GPT-based model, Duolando, which autoregressively predicts the subsequent tokenized motion conditioned on the coordinated information of the music, the leader's and the follower's movements.
  arXiv  Detail & Related papers  (2024-03-27T17:57:02Z)
• Controllable Group Choreography using Contrastive Diffusion [9.524877757674176]
  Music-driven group choreography holds significant potential for a wide range of industrial applications. We introduce a Group Contrastive Diffusion (GCD) strategy to enhance the connection between dancers and their group. We demonstrate the effectiveness of our approach in producing visually captivating and consistent group dance motions.
  arXiv  Detail & Related papers  (2023-10-29T11:59:12Z)
• Quantized GAN for Complex Music Generation from Dance Videos [48.196705493763986]
  We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates musical samples conditioned on dance videos. Our proposed framework takes dance video frames and human body motion as input, and learns to generate music samples that plausibly accompany the corresponding input.
  arXiv  Detail & Related papers  (2022-04-01T17:53:39Z)
• Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory [92.81383016482813]
  We propose a novel music-to-dance framework, Bailando, for driving 3D characters to dance following a piece of music. We introduce an actor-critic Generative Pre-trained Transformer (GPT) that composes units to a fluent dance coherent to the music. Our proposed framework achieves state-of-the-art performance both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-03-24T13:06:43Z)
• Music-to-Dance Generation with Optimal Transport [48.92483627635586]
  We propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning to generate 3D dance choreographs from music. We introduce an optimal transport distance for evaluating the authenticity of the generated dance distribution and a Gromov-Wasserstein distance to measure the correspondence between the dance distribution and the input music.
  arXiv  Detail & Related papers  (2021-12-03T09:37:26Z)
• Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structure [47.09425316677689]
  We present a music-driven motion synthesis framework that generates long-term sequences of human motions synchronized with the input beats. Our framework enables generation of diverse motions that are controlled by the content of the music, and not only by the beat.
  arXiv  Detail & Related papers  (2021-11-23T21:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.