PirouNet: Creating Intentional Dance with Semi-Supervised Conditional Recurrent Variational Autoencoders

• URL: http://arxiv.org/abs/2207.12126v1
• Date: Thu, 21 Jul 2022 18:04:59 GMT
• Title: PirouNet: Creating Intentional Dance with Semi-Supervised Conditional Recurrent Variational Autoencoders
• Authors: Mathilde Papillon, Mariel Pettee, Nina Miolane
• Abstract summary: We propose "PirouNet", a semi-supervised conditional recurrent variational autoencoder with a dance labeling web application. Thanks to the proposed semi-supervised approach, PirouNet only requires a small portion of the dataset to be labeled, typically on the order of 1%. We extensively evaluate PirouNet's dance creations through a series of qualitative and quantitative metrics, validating its applicability as a tool for choreographers.
• Score: 3.867363075280544
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Using Artificial Intelligence (AI) to create dance choreography with intention is still at an early stage. Methods that conditionally generate dance sequences remain limited in their ability to follow choreographer-specific creative intentions, often relying on external prompts or supervised learning. In the same vein, fully annotated dance datasets are rare and labor intensive. To fill this gap and help leverage deep learning as a meaningful tool for choreographers, we propose "PirouNet", a semi-supervised conditional recurrent variational autoencoder together with a dance labeling web application. PirouNet allows dance professionals to annotate data with their own subjective creative labels and subsequently generate new bouts of choreography based on their aesthetic criteria. Thanks to the proposed semi-supervised approach, PirouNet only requires a small portion of the dataset to be labeled, typically on the order of 1%. We demonstrate PirouNet's capabilities as it generates original choreography based on the "Laban Time Effort", an established dance notion describing intention for a movement's time dynamics. We extensively evaluate PirouNet's dance creations through a series of qualitative and quantitative metrics, validating its applicability as a tool for choreographers.

Related papers

• Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns [48.54956784928394]
  Lodge++ is a choreography framework to generate high-quality, ultra-long, and vivid dances given the music and desired genre. To handle the challenges in computational efficiency, Lodge++ adopts a two-stage strategy to produce dances from coarse to fine. Lodge++ is validated by extensive experiments, which show that our method can rapidly generate ultra-long dances suitable for various dance genres.
  arXiv  Detail & Related papers  (2024-10-27T09:32:35Z)
• Automatic Dance Video Segmentation for Understanding Choreography [10.053913399613764]
  We propose a method to automatically segment a dance video into each movement. To build our training dataset, we annotate segmentation points to dance videos in the AIST Dance Video Database. The evaluation study shows that the proposed method can estimate segmentation points with high accuracy.
  arXiv  Detail & Related papers  (2024-05-30T06:19:01Z)
• Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment [87.20240797625648]
  We introduce a novel task within the field of 3D dance generation, termed dance accompaniment. It requires the generation of responsive movements from a dance partner, the "follower", synchronized with the lead dancer's movements and the underlying musical rhythm. We propose a GPT-based model, Duolando, which autoregressively predicts the subsequent tokenized motion conditioned on the coordinated information of the music, the leader's and the follower's movements.
  arXiv  Detail & Related papers  (2024-03-27T17:57:02Z)
• Intentional Choreography with Semi-Supervised Recurrent VAEs [3.867363075280544]
  We present PirouNet, a semi-supervised recurrent variational autoencoder. We show that PirouNet conditionally generates dance sequences in the style of the choreographer.
  arXiv  Detail & Related papers  (2022-09-20T21:38:51Z)
• BRACE: The Breakdancing Competition Dataset for Dance Motion Synthesis [123.73677487809418]
  We introduce a new dataset aiming to challenge common assumptions in dance motion synthesis. We focus on breakdancing which features acrobatic moves and tangled postures. Our efforts produced the BRACE dataset, which contains over 3 hours and 30 minutes of densely annotated poses.
  arXiv  Detail & Related papers  (2022-07-20T18:03:54Z)
• Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory [92.81383016482813]
  We propose a novel music-to-dance framework, Bailando, for driving 3D characters to dance following a piece of music. We introduce an actor-critic Generative Pre-trained Transformer (GPT) that composes units to a fluent dance coherent to the music. Our proposed framework achieves state-of-the-art performance both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-03-24T13:06:43Z)
• Music-to-Dance Generation with Optimal Transport [48.92483627635586]
  We propose a Music-to-Dance with Optimal Transport Network (MDOT-Net) for learning to generate 3D dance choreographs from music. We introduce an optimal transport distance for evaluating the authenticity of the generated dance distribution and a Gromov-Wasserstein distance to measure the correspondence between the dance distribution and the input music.
  arXiv  Detail & Related papers  (2021-12-03T09:37:26Z)
• Feel The Music: Automatically Generating A Dance For An Input Song [58.653867648572]
  We present a general computational approach that enables a machine to generate a dance for any input music. We encode intuitive, flexibles for what a 'good' dance is: the structure of the dance should align with the structure of the music.
  arXiv  Detail & Related papers  (2020-06-21T20:29:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.