MultiAct: Text-to-Motion Generation from Composite Text via Tailored Attention Guidance

• URL: http://arxiv.org/abs/2605.30925v1
• Date: Fri, 29 May 2026 07:14:48 GMT
• Title: MultiAct: Text-to-Motion Generation from Composite Text via Tailored Attention Guidance
• Abstract summary: MultiAct is an unpaired, inference-time framework for compositional text-to-motion synthesis.<n>Our method counteracts semantic collapse by adaptively amplifying cross-attention scores associated with underrepresented prompt components.
• Score: 54.131431576597514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-motion generation has progressed rapidly in recent years, offering an expressive interface for animation and human-computer interaction. However, current models remain brittle when handling prompts that describe multiple actions occurring at the same time. Rather than realizing all components of a composite description, models frequently prioritize a single dominant action and neglect the rest, leading to incomplete or ambiguous motion. We present MultiAct, an unpaired, inference-time framework for compositional text-to-motion synthesis that operates directly on pretrained motion generators without retraining or architectural modification. Our method counteracts semantic collapse by adaptively amplifying cross-attention scores associated with underrepresented prompt components. We note that effective modulation depends on prompt-specific choices, such as which tokens and layers to target, and introduce a lightweight auxiliary decision scheme that determines the most effective attention-strengthening parametrization. Extensive quantitative and qualitative evaluations demonstrate that MultiAct consistently outperforms existing baselines on composite prompts, achieving improved semantic coverage while preserving motion realism. Project page: https://natsala13.github.io/multiact.github.io.

Related papers

• Video-Text Temporal Localization via Multi-Scale Convolution and Dynamic Routing [7.704586969939547]
  Video-text temporal localization requires precise alignment between natural language queries and corresponding video segments.<n>We present a novel framework that addresses two critical limitations of existing methods.<n>Our results validate that combining hierarchical temporal modeling with structured semantic routing provides an effective solution for fine-grained video-language understanding.
  arXiv  Detail & Related papers  (2026-07-06T13:52:46Z)
• Motion-Adapter: A Diffusion Model Adapter for Text-to-Motion Generation of Compound Actions [13.380249866020355]
  We propose a plug-and-play module that guides text-to-motion diffusion models in generating compound actions.<n>Our method consistently produces more faithful and coherent compound motions across diverse textual prompts.
  arXiv  Detail & Related papers  (2026-04-17T15:08:17Z)
• InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidance [20.740979380270126]
  We propose InterDyad, a synthesis framework for interactive dyadic dynamics.<n>We first design an Interactivity that achieves video reenactment based on identity-agnostic motion priors extracted from reference videos.<n>By leveraging a Multimodal Large Language Model (MLLM), our framework is able to distill linguistic intent from audio to dictate the precise timing and appropriateness of reactions.<n> Comprehensive experiments demonstrate that InterDyad significantly outperforms state-of-the-art methods in producing natural and contextually grounded two-person interactions.
  arXiv  Detail & Related papers  (2026-03-24T12:27:52Z)
• MIBURI: Towards Expressive Interactive Gesture Synthesis [62.45332399212876]
  Embodied Conversational Agents (ECAs) aim to emulate human face-to-face interaction through speech, gestures, and facial expressions.<n>Existing solutions for ECAs produce rigid, low-diversity motions that are unsuitable for human-like interaction.<n>We present MIBURI, the first online, causal framework for generating expressive full-body gestures and facial expressions synchronized with real-time spoken dialogue.
  arXiv  Detail & Related papers  (2026-03-03T18:59:51Z)
• HINT: Hierarchical Interaction Modeling for Autoregressive Multi-Human Motion Generation [55.73037290387896]
  We introduce HINT, the first autoregressive framework for multi-human motion generation with Hierarchical INTeraction modeling in diffusion.<n>First, HINT leverages a disentangled motion representation within a canonicalized latent space, decoupling local motion semantics from inter-person interactions.<n>Second, HINT adopts a sliding-window strategy for efficient online generation, and aggregates local within-window and global cross-window conditions to capture past human history, inter-person dependencies, and align with text guidance.
  arXiv  Detail & Related papers  (2026-01-28T08:47:23Z)
• MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing [53.98607267063729]
  MotionVerse is a framework to comprehend, generate, and edit human motion in both single-person and multi-person scenarios.<n>We employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens.<n>We also introduce a textitDelay Parallel Modeling strategy, which temporally staggers the encoding of residual token streams.
  arXiv  Detail & Related papers  (2025-09-28T04:20:56Z)
• CASIM: Composite Aware Semantic Injection for Text to Motion Generation [15.53049009014166]
  We propose a composite-aware semantic injection mechanism that learns the dynamic correspondence between text and motion tokens.<n> Experiments on HumanML3D and KIT benchmarks demonstrate that CASIM consistently improves motion quality, text-motion alignment, and retrieval scores.
  arXiv  Detail & Related papers  (2025-02-04T07:22:07Z)
• OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers [45.808597624491156]
  We present OMG, a novel framework, which enables compelling motion generation from zero-shot open-vocabulary text prompts. At the pre-training stage, our model improves the generation ability by learning the rich out-of-domain inherent motion traits. At the fine-tuning stage, we introduce motion ControlNet, which incorporates text prompts as conditioning information.
  arXiv  Detail & Related papers  (2023-12-14T14:31:40Z)
• SemanticBoost: Elevating Motion Generation with Augmented Textual Cues [73.83255805408126]
  Our framework comprises a Semantic Enhancement module and a Context-Attuned Motion Denoiser (CAMD) The CAMD approach provides an all-encompassing solution for generating high-quality, semantically consistent motion sequences. Our experimental results demonstrate that SemanticBoost, as a diffusion-based method, outperforms auto-regressive-based techniques.
  arXiv  Detail & Related papers  (2023-10-31T09:58:11Z)
• TEACH: Temporal Action Composition for 3D Humans [50.97135662063117]
  Given a series of natural language descriptions, our task is to generate 3D human motions that correspond semantically to the text. In particular, our goal is to enable the synthesis of a series of actions, which we refer to as temporal action composition.
  arXiv  Detail & Related papers  (2022-09-09T00:33:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.