Text-driven Motion Generation: Overview, Challenges and Directions

• URL: http://arxiv.org/abs/2505.09379v1
• Date: Wed, 14 May 2025 13:33:12 GMT
• Title: Text-driven Motion Generation: Overview, Challenges and Directions
• Authors: Ali Rida Sahili, Najett Neji, Hedi Tabia,
• Abstract summary: Text-driven motion generation offers a powerful and intuitive way to create human movements directly from natural language.<n>This makes it especially useful in areas like virtual reality, gaming, human-computer interaction, and robotics.<n>We aim to capture where the field currently stands, bring attention to its key challenges and limitations, and highlight promising directions for future exploration.
• Score: 5.292618442300405
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-driven motion generation offers a powerful and intuitive way to create human movements directly from natural language. By removing the need for predefined motion inputs, it provides a flexible and accessible approach to controlling animated characters. This makes it especially useful in areas like virtual reality, gaming, human-computer interaction, and robotics. In this review, we first revisit the traditional perspective on motion synthesis, where models focused on predicting future poses from observed initial sequences, often conditioned on action labels. We then provide a comprehensive and structured survey of modern text-to-motion generation approaches, categorizing them from two complementary perspectives: (i) architectural, dividing methods into VAE-based, diffusion-based, and hybrid models; and (ii) motion representation, distinguishing between discrete and continuous motion generation strategies. In addition, we explore the most widely used datasets, evaluation methods, and recent benchmarks that have shaped progress in this area. With this survey, we aim to capture where the field currently stands, bring attention to its key challenges and limitations, and highlight promising directions for future exploration. We hope this work offers a valuable starting point for researchers and practitioners working to push the boundaries of language-driven human motion synthesis.

Related papers

• Motion Generation: A Survey of Generative Approaches and Benchmarks [1.4254358932994455]
  We provide an in-depth categorization of motion generation methods based on their underlying generative strategies.<n>Our main focus is on papers published in top-tier venues since 2023, reflecting the most recent advancements in the field.<n>We analyze architectural principles, conditioning mechanisms, and generation settings, and compile a detailed overview of the evaluation metrics and datasets used across the literature.
  arXiv  Detail & Related papers  (2025-07-07T19:04:56Z)
• UniHM: Universal Human Motion Generation with Object Interactions in Indoor Scenes [26.71077287710599]
  We propose UniHM, a unified motion language model that leverages diffusion-based generation for scene-aware human motion.<n>UniHM is the first framework to support both Text-to-Motion and Text-to-Human-Object Interaction (HOI) in complex 3D scenes.<n>Our approach introduces three key contributions: (1) a mixed-motion representation that fuses continuous 6DoF motion with discrete local motion tokens to improve motion realism; (2) a novel Look-Up-Free Quantization VAE that surpasses traditional VQ-VAEs in both reconstruction accuracy and generative performance; and (3) an enriched version of
  arXiv  Detail & Related papers  (2025-05-19T07:02:12Z)
• Motion Generation Review: Exploring Deep Learning for Lifelike Animation with Manifold [4.853986914715961]
  Human motion generation involves creating natural sequences of human body poses, widely used in gaming, virtual reality, and human-computer interaction.<n>Previous work has focused on motion generation based on signals like movement, music, text, or scene background.<n>Mandela learning offers a solution by reducing data dimensionality and capturing subspaces of effective motion.
  arXiv  Detail & Related papers  (2024-12-12T08:27:15Z)
• KinMo: Kinematic-aware Human Motion Understanding and Generation [6.962697597686156]
  Current human motion synthesis frameworks rely on global action descriptions.<n>A single coarse description fails to capture details like variations in speed, limb positioning, and kinematic dynamics.<n>We introduce textbfKinMo, a unified framework built on a hierarchical describable motion representation.
  arXiv  Detail & Related papers  (2024-11-23T06:50:11Z)
• Joint-Dataset Learning and Cross-Consistent Regularization for Text-to-Motion Retrieval [4.454835029368504]
  We focus on the recently introduced text-motion retrieval which aim to search for sequences that are most relevant to a natural motion description. Despite recent efforts to explore these promising avenues, a primary challenge remains the insufficient data available to train robust text-motion models. We propose to investigate joint-dataset learning - where we train on multiple text-motion datasets simultaneously. We also introduce a transformer-based motion encoder, called MoT++, which employs the specified-temporal attention to process sequences of skeleton data.
  arXiv  Detail & Related papers  (2024-07-02T09:43:47Z)
• Learning Generalizable Human Motion Generator with Reinforcement Learning [95.62084727984808]
  Text-driven human motion generation is one of the vital tasks in computer-aided content creation. Existing methods often overfit specific motion expressions in the training data, hindering their ability to generalize. We present textbfInstructMotion, which incorporate the trail and error paradigm in reinforcement learning for generalizable human motion generation.
  arXiv  Detail & Related papers  (2024-05-24T13:29:12Z)
• Generating Human Interaction Motions in Scenes with Text Control [66.74298145999909]
  We present TeSMo, a method for text-controlled scene-aware motion generation based on denoising diffusion models. Our approach begins with pre-training a scene-agnostic text-to-motion diffusion model. To facilitate training, we embed annotated navigation and interaction motions within scenes.
  arXiv  Detail & Related papers  (2024-04-16T16:04:38Z)
• Priority-Centric Human Motion Generation in Discrete Latent Space [59.401128190423535]
  We introduce a Priority-Centric Motion Discrete Diffusion Model (M2DM) for text-to-motion generation. M2DM incorporates a global self-attention mechanism and a regularization term to counteract code collapse. We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token.
  arXiv  Detail & Related papers  (2023-08-28T10:40:16Z)
• Human Motion Generation: A Survey [67.38982546213371]
  Human motion generation aims to generate natural human pose sequences and shows immense potential for real-world applications. Most research within this field focuses on generating human motions based on conditional signals, such as text, audio, and scene contexts. We present a comprehensive literature review of human motion generation, which is the first of its kind in this field.
  arXiv  Detail & Related papers  (2023-07-20T14:15:20Z)
• Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations [61.659439423703155]
  TOHO: Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations. Our method generates continuous motions that are parameterized only by the temporal coordinate. This work takes a step further toward general human-scene interaction simulation.
  arXiv  Detail & Related papers  (2023-03-23T09:31:56Z)
• TEMOS: Generating diverse human motions from textual descriptions [53.85978336198444]
  We address the problem of generating diverse 3D human motions from textual descriptions. We propose TEMOS, a text-conditioned generative model leveraging variational autoencoder (VAE) training with human motion data. We show that TEMOS framework can produce both skeleton-based animations as in prior work, as well more expressive SMPL body motions.
  arXiv  Detail & Related papers  (2022-04-25T14:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.