Exploring Motion-Language Alignment for Text-driven Motion Generation

• URL: http://arxiv.org/abs/2604.02973v1
• Date: Fri, 03 Apr 2026 11:24:34 GMT
• Title: Exploring Motion-Language Alignment for Text-driven Motion Generation
• Abstract summary: We propose a framework that integrates global motion priors with fine-grained local conditioning.<n>This design enables the model to capture common motion patterns, while establishing detailed alignment between texts and motions.
• Score: 40.39333668557699
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-driven human motion generation aims to synthesize realistic motion sequences that follow textual descriptions. Despite recent advances, accurately aligning motion dynamics with textual semantics remains a fundamental challenge. In this paper, we revisit text-to-motion generation from the perspective of motion-language alignment and propose MLA-Gen, a framework that integrates global motion priors with fine-grained local conditioning. This design enables the model to capture common motion patterns, while establishing detailed alignment between texts and motions. Furthermore, we identify a previously overlooked attention sink phenomenon in human motion generation, where attention disproportionately concentrates on the start text token, limiting the utilization of informative textual cues and leading to degraded semantic grounding. To analyze this issue, we introduce SinkRatio, a metric for measuring attention concentration, and develop alignment-aware masking and control strategies to regulate attention during generation. Extensive experiments demonstrate that our approach consistently improves both motion quality and motion-language alignment over strong baselines. Code will be released upon acceptance.

Related papers

• MotionMERGE: A Multi-granular Framework for Human Motion Editing, Reasoning, Generation, and Explanation [66.66098171359995]
  MotionMERGE is a unified framework that bridges the granularity gap in motion-language models.<n>First, we pioneer the study of fine-grained language-guided motion control, including detailed understanding and localized editing.<n>Second, we design ReasoningAware Granularity-Synergy pre-training, a novel strategy that employs joint supervision for cross-granularity alignment.<n>Third, we curate MotionFineEdit, a large-scale dataset with the first finegrained-temporal corrective instructions and motion-grounded CoT annotations.
  arXiv  Detail & Related papers  (2026-05-18T18:00:04Z)
• Beyond Global Alignment: Fine-Grained Motion-Language Retrieval via Pyramidal Shapley-Taylor Learning [56.6025512458557]
  Motion-language retrieval aims to bridge the semantic gap between natural language and human motion.<n>Existing approaches predominantly focus on aligning entire motion sequences with global textual representations.<n>We propose a novel Pyramidal Shapley-Taylor (PST) learning framework for fine-grained motion-language retrieval.
  arXiv  Detail & Related papers  (2026-01-29T16:00:12Z)
• Text-driven Motion Generation: Overview, Challenges and Directions [5.292618442300405]
  Text-driven motion generation offers a powerful and intuitive way to create human movements directly from natural language.<n>This makes it especially useful in areas like virtual reality, gaming, human-computer interaction, and robotics.<n>We aim to capture where the field currently stands, bring attention to its key challenges and limitations, and highlight promising directions for future exploration.
  arXiv  Detail & Related papers  (2025-05-14T13:33:12Z)
• CASIM: Composite Aware Semantic Injection for Text to Motion Generation [15.53049009014166]
  We propose a composite-aware semantic injection mechanism that learns the dynamic correspondence between text and motion tokens.<n> Experiments on HumanML3D and KIT benchmarks demonstrate that CASIM consistently improves motion quality, text-motion alignment, and retrieval scores.
  arXiv  Detail & Related papers  (2025-02-04T07:22:07Z)
• KinMo: Kinematic-aware Human Motion Understanding and Generation [6.962697597686156]
  Current human motion synthesis frameworks rely on global action descriptions.<n>A single coarse description, such as run, fails to capture details such as variations in speed, limb positioning, and kinematic dynamics.<n>We introduce KinMo, a unified framework built on a hierarchical describable motion representation.
  arXiv  Detail & Related papers  (2024-11-23T06:50:11Z)
• Chronologically Accurate Retrieval for Temporal Grounding of Motion-Language Models [12.221087476416056]
  We propose Chronologically Accurate Retrieval to evaluate the chronological understanding of motion-language models. We decompose textual descriptions into events, and prepare negative text samples by shuffling the order of events in compound action descriptions. We then design a simple task for motion-language models to retrieve the more likely text from the ground truth and its chronologically shuffled version.
  arXiv  Detail & Related papers  (2024-07-22T06:25:21Z)
• Learning Generalizable Human Motion Generator with Reinforcement Learning [95.62084727984808]
  Text-driven human motion generation is one of the vital tasks in computer-aided content creation. Existing methods often overfit specific motion expressions in the training data, hindering their ability to generalize. We present textbfInstructMotion, which incorporate the trail and error paradigm in reinforcement learning for generalizable human motion generation.
  arXiv  Detail & Related papers  (2024-05-24T13:29:12Z)
• SemanticBoost: Elevating Motion Generation with Augmented Textual Cues [73.83255805408126]
  Our framework comprises a Semantic Enhancement module and a Context-Attuned Motion Denoiser (CAMD) The CAMD approach provides an all-encompassing solution for generating high-quality, semantically consistent motion sequences. Our experimental results demonstrate that SemanticBoost, as a diffusion-based method, outperforms auto-regressive-based techniques.
  arXiv  Detail & Related papers  (2023-10-31T09:58:11Z)
• HumanTOMATO: Text-aligned Whole-body Motion Generation [30.729975715600627]
  This work targets a novel text-driven whole-body motion generation task. It aims at generating high-quality, diverse, and coherent facial expressions, hand gestures, and body motions simultaneously.
  arXiv  Detail & Related papers  (2023-10-19T17:59:46Z)
• Priority-Centric Human Motion Generation in Discrete Latent Space [59.401128190423535]
  We introduce a Priority-Centric Motion Discrete Diffusion Model (M2DM) for text-to-motion generation. M2DM incorporates a global self-attention mechanism and a regularization term to counteract code collapse. We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token.
  arXiv  Detail & Related papers  (2023-08-28T10:40:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.