MUGEN: A Unified Framework for Efficient Motion Understanding and Generation

• URL: http://arxiv.org/abs/2607.27581v1
• Date: Thu, 30 Jul 2026 01:55:51 GMT
• Title: MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
• Abstract summary: We propose a unified motion--language framework that pays neither cost: no codebook, one draw.<n>A single adaptive-length autoencoder compresses any-length motion into a few continuous latent slots, the system's only motion representation.<n> MUGEN leads language-model baselines on FID on HumanML3D while raising retrieval precision above the real-motion reference.
• Score: 17.452895169020817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Grounding human motion in language, and language in motion, is a central step toward physical AI systems that can understand, generate, and communicate human behavior. Unified motion--language systems first coupled the two directions through a shared discrete motion codebook, but quantization limits generation quality. The strongest generators buy quality back at growing cost: stacked residual codebooks enlarge the representation; masked decoding stages, long autoregressive rollouts, and denoising chains of tens to hundreds of steps stretch inference; even the continuous-latent designs among them reach their latent only through an iterative diffusion head; and none of this decoding machinery serves understanding. We therefore propose MUGEN, a unified motion--language framework that pays neither cost: no codebook, one draw. A single adaptive-length autoencoder compresses any-length motion into a few continuous latent slots, the system's only motion representation: the language model generates them for text-to-motion and reads them back for motion understanding. Depth-routed hidden states let each slot read from the transformer depth it needs, and a calibrated head predicts a joint distribution over the full latent set, so a single draw carries the text-conditional, cross-slot variation a description permits. At a decoding cost of K language-model steps, one draw, and one decoder pass, MUGEN leads language-model baselines on FID on HumanML3D while raising retrieval precision above the real-motion reference under the standard evaluator, achieves the best CIDEr and BLEU@4 scores, and surpasses the discrete-token state of the art on every retrieval and alignment metric on SnapMoGen.

Related papers

• SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling [16.84657295044791]
  We introduce SeMoCo, a semantic-first motion, together with a dual-axis motion generator for language-conditioned motion generation.<n>We also construct $$-MotionVerse, a large-scale, multi-source human-motion dataset unified under the SOMA representation.
  arXiv  Detail & Related papers  (2026-08-25T09:58:20Z)
• CLAW: Composable Language-Annotated Whole-body Motion Generation [55.99805728566105]
  We present CLAW, a pipeline for scalable generation of language-annotated whole-body motion data.<n>CLAW composes motion primitives from a kinematic planner, parameterized by movement, heading, speed, pelvis height, and duration.<n>A low-level controller tracks these references in MuJoCo simulation, yielding physically grounded trajectories.
  arXiv  Detail & Related papers  (2026-04-13T10:02:04Z)
• SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning [54.232148007248874]
  Current Sign Language Production (SLP) frameworks face a stark trade-off: direct text-to-pose models suffer from regression-to-the-mean effects.<n>We propose a novel training paradigm that leverages sparses to capture the true underlying distribution of human signing.<n>By predicting dense motion from these discrete anchors, our approach mitigates regression-to-the-mean while ensuring fluid articulation.
  arXiv  Detail & Related papers  (2026-03-11T06:02:36Z)
• LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokens [19.167250154665812]
  We propose LLaMo, a framework that extends pretrained large language models through a modality-specific Mixture-of-Transformers architecture.<n>We encode human motion into a causal continuous latent space and maintain the next-token prediction paradigm in the decoder-only backbone.<n>Our experiments demonstrate that LLaMo achieves high-fidelity text-to-motion generation and motion-to-text captioning in general settings.
  arXiv  Detail & Related papers  (2026-02-12T20:02:21Z)
• Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenization [20.063863466319326]
  SignViP is a novel framework that incorporates multiple fine-grained conditions for improved generation fidelity.<n>SignViP achieves state-of-the-art performance across metrics, including video quality temporal coherence, and semantic fidelity.
  arXiv  Detail & Related papers  (2025-06-19T02:56:06Z)
• MAG: Multi-Modal Aligned Autoregressive Co-Speech Gesture Generation without Vector Quantization [8.605691647343065]
  This work focuses on full-body co-speech gesture generation. Existing methods typically employ an autoregressive model accompanied by vector-quantized tokens for gesture generation.<n>We propose MAG, a novel multi-modal aligned framework for high-quality and diverse co-speech gesture synthesis without relying on discrete tokenization.
  arXiv  Detail & Related papers  (2025-03-18T09:02:02Z)
• Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs [67.59291068131438]
  Motion-Agent is a conversational framework designed for general human motion generation, editing, and understanding. Motion-Agent employs an open-source pre-trained language model to develop a generative agent, MotionLLM, that bridges the gap between motion and text.
  arXiv  Detail & Related papers  (2024-05-27T09:57:51Z)
• Seamless Human Motion Composition with Blended Positional Encodings [38.85158088021282]
  We introduce FlowMDM, the first diffusion-based model that generates seamless Human Motion Compositions (HMC) without postprocessing or redundant denoising steps. We achieve state-of-the-art results in terms of accuracy, realism, and smoothness on the Babel and HumanML3D datasets.
  arXiv  Detail & Related papers  (2024-02-23T18:59:40Z)
• Kosmos-G: Generating Images in Context with Multimodal Large Language Models [117.0259361818715]
  Current subject-driven image generation methods require test-time tuning and cannot accept interleaved multi-image and text input. This paper presents Kosmos-G, a model that leverages the advanced multimodal perception capabilities of Multimodal Large Language Models. Kosmos-G demonstrates an impressive capability of zero-shot subject-driven generation with interleaved multi-image and text input.
  arXiv  Detail & Related papers  (2023-10-04T17:28:44Z)
• DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusion [70.33381660741861]
  We present DiverseMotion, a new approach for synthesizing high-quality human motions conditioned on textual descriptions. We show that our DiverseMotion achieves the state-of-the-art motion quality and competitive motion diversity.
  arXiv  Detail & Related papers  (2023-09-04T05:43:48Z)
• Priority-Centric Human Motion Generation in Discrete Latent Space [59.401128190423535]
  We introduce a Priority-Centric Motion Discrete Diffusion Model (M2DM) for text-to-motion generation. M2DM incorporates a global self-attention mechanism and a regularization term to counteract code collapse. We also present a motion discrete diffusion model that employs an innovative noise schedule, determined by the significance of each motion token.
  arXiv  Detail & Related papers  (2023-08-28T10:40:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.