ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation

• URL: http://arxiv.org/abs/2605.11704v1
• Date: Tue, 12 May 2026 07:58:58 GMT
• Title: ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation
• Abstract summary: ScaleMoGen is a scale-wise autoregressive framework for text-driven human motion generation.<n>We quantize 3D motions into compositional discrete tokens across multiple skeletal-emporal scales.<n>Our skeletal-temporal multi-scale representation naturally facilitates training-free, text-guided motion editing.
• Score: 21.04046292025699
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present ScaleMoGen, a scale-wise autoregressive framework for text-driven human motion generation. Unlike conventional autoregressive approaches that rely on standard next-token prediction, ScaleMoGen frames motion generation as a coarse-to-fine process. We quantize 3D motions into compositional discrete tokens across multiple skeletal-emporal scales of increasing granularity, learning to generate motion by autoregressively predicting next-scale token maps. To maintain structural integrity, our motion tokenizers and quantizers are explicitly designed so that discrete tokens at every scale strictly preserve the skeletal hierarchy. Additionally, we employ bitwise quantization and prediction, which efficiently scale up the tokenizer vocabulary to preserve motion details and stabilize optimization. Extensive experiments demonstrate that ScaleMoGen achieves state-of-the-art performance, establishing an FID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Score of 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen dataset. Furthermore, we demonstrate that our skeletal-temporal multi-scale representation naturally facilitates training-free, text-guided motion editing.

Related papers

• MotionMAR: Multi-scale Auto-Regressive Human Motion Reconstruction from Sparse Observations [51.837372131319285]
  MotionMAR is a framework for motion reconstruction from sparse observations.<n>It first estimates the global trajectory of human motion and then gradually refines the temporal details.<n>MotionMAR achieves state-of-the-art accuracy on the AMASS dataset.
  arXiv  Detail & Related papers  (2026-06-22T08:15:56Z)
• MotionVLA: Vision-Language-Action Model for Humanoid Motion [54.785960777274276]
  We present DSFT, a dual-stream frequency tokenizer that separates motion into Base and physical streams.<n>We also present MotionVLA, a Qwen3.5-based model that arranges Base and physical tokens in a unified sequence.
  arXiv  Detail & Related papers  (2026-06-13T06:10:48Z)
• AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling [60.13338774249153]
  We introduce OmniHuMo, a dataset of over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations.<n>We propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer.<n>Experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.
  arXiv  Detail & Related papers  (2026-05-28T07:15:19Z)
• Next-Scale Autoregressive Models for Text-to-Motion Generation [35.54703580430989]
  MoScale is a next-scale AR framework that generates motion hierarchically from coarse to fine temporal resolutions.<n>MoScale achieves SOTA text-to-motion performance with high training efficiency, scales effectively with model size, and generalizes zero-shot to diverse motion generation and editing tasks.
  arXiv  Detail & Related papers  (2026-04-04T17:07:37Z)
• HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation [63.04826523091837]
  HY-Motion 1.0 is a series of state-of-the-art, large-scale, motion generation models capable of generating 3D human motions from textual descriptions.<n>We introduce a comprehensive, full-stage training paradigm -- including large-scale pretraining on over 3,000 hours of motion data.<n>Our model achieves the most extensive coverage, spanning over 200 motion categories across 6 major classes.
  arXiv  Detail & Related papers  (2025-12-29T13:46:24Z)
• MoSa: Motion Generation with Scalable Autoregressive Modeling [41.75643989500953]
  We introduce MoSa, a novel hierarchical motion generation framework for text-driven 3D human motion generation.<n>MoSa achieves state-of-the-art generation quality and efficiency, outperforming prior methods in both fidelity and speed.<n>MoSa generalizes well to downstream tasks such as motion editing, requiring no additional fine-tuning.
  arXiv  Detail & Related papers  (2025-11-03T03:47:58Z)
• OmniMotion-X: Versatile Multimodal Whole-Body Motion Generation [52.579531290307926]
  This paper introduces OmniMotion-X, a versatile framework for whole-body human motion generation.<n> OmniMotion-X efficiently supports diverse multimodal tasks, including text-to-motion, music-to-dance, speech-to-gesture.<n>To enable high-quality multimodal training, we construct OmniMoCap-X, the largest unified multimodal motion dataset to date.
  arXiv  Detail & Related papers  (2025-10-22T17:25:33Z)
• Puppeteer: Rig and Animate Your 3D Models [105.11046762553121]
  Puppeteer is a comprehensive framework that addresses both automatic rigging and animation for diverse 3D objects.<n>Our system first predicts plausible skeletal structures via an auto-regressive transformer.<n>It then infers skinning weights via an attention-based architecture.
  arXiv  Detail & Related papers  (2025-08-14T17:59:31Z)
• Multi-Modal Graph Convolutional Network with Sinusoidal Encoding for Robust Human Action Segmentation [10.122882293302787]
  temporal segmentation of human actions is critical for intelligent robots in collaborative settings.<n>We propose a Multi-Modal Graph Convolutional Network (MMGCN) that integrates low-frame-rate (e.g., 1 fps) visual data with high-frame-rate (e.g., 30 fps) motion data.<n>Our approach outperforms state-of-the-art methods, especially in action segmentation accuracy.
  arXiv  Detail & Related papers  (2025-07-01T13:55:57Z)
• Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer [62.29951737214263]
  Existing algorithms directly generate the full sequence which is expensive and prone to errors. We propose KeyMotion, that generates plausible human motion sequences corresponding to input text. We use a Variationalcoder (VAE) with Kullback-Leibler regularization to project the Autoencoder into a latent space. For the reverse diffusion, we propose a novel Parallel Skip Transformer that performs cross-modal attention between the design latents and text condition.
  arXiv  Detail & Related papers  (2024-05-24T11:12:37Z)
• BAMM: Bidirectional Autoregressive Motion Model [14.668729995275807]
  Bidirectional Autoregressive Motion Model (BAMM) is a novel text-to-motion generation framework. BAMM consists of two key components: a motion tokenizer that transforms 3D human motion into discrete tokens in latent space, and a masked self-attention transformer that autoregressively predicts randomly masked tokens. This feature enables BAMM to simultaneously achieving high-quality motion generation with enhanced usability and built-in motion editability.
  arXiv  Detail & Related papers  (2024-03-28T14:04:17Z)
• MoMask: Generative Masked Modeling of 3D Human Motions [25.168781728071046]
  MoMask is a novel framework for text-driven 3D human motion generation. A hierarchical quantization scheme is employed to represent human motion as discrete motion tokens. MoMask outperforms state-of-art methods on the text-to-motion generation task.
  arXiv  Detail & Related papers  (2023-11-29T19:04:10Z)
• TapMo: Shape-aware Motion Generation of Skeleton-free Characters [64.83230289993145]
  We present TapMo, a Text-driven Animation Pipeline for Motion in a broad spectrum of skeleton-free 3D characters. TapMo comprises two main components - Mesh Handle Predictor and Shape-aware Diffusion Module.
  arXiv  Detail & Related papers  (2023-10-19T12:14:32Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.