AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling

• URL: http://arxiv.org/abs/2605.29488v2
• Date: Sun, 31 May 2026 04:55:43 GMT
• Title: AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling
• Abstract summary: We introduce OmniHuMo, a dataset of over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations.<n>We propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer.<n>Experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.
• Score: 60.13338774249153
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditional human motion generation remains a fundamental challenge in computer vision and robotics. Despite significant progress, current methods are often constrained by fixed modality configurations and task-specific architectures, leaving cross-modal interactions and the scaling laws of multimodal-conditioned synthesis largely underexplored. A key bottleneck is the scarcity of large-scale modality-aligned motion data, limiting generalization across diverse control signals. In this work, we introduce OmniHuMo, a large-scale, high-quality dataset comprising over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations (e.g., text, speech, music, and trajectory). Leveraging OmniHuMo, we propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer, enabling high-quality motion synthesis under arbitrary modality combinations. Extensive experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.

Related papers

• OMG: Omni-Modal Motion Generation for Generalist Humanoid Control [30.236954254902614]
  We argue that the key to general-purpose humanoid control is to build a scalable brain, atop a reactive motion tracking system.<n>We present OMG, which addresses two challenges: acquiring a vast amount of high-quality data to achieve general purpose control, and equipping the generator with the capability to condition on compositional, multi-modal inputs.
  arXiv  Detail & Related papers  (2026-06-09T02:46:14Z)
• CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration [66.81078857385698]
  CoMoGen is a controllable video generation framework that generates realistic interactive dynamics from a single binary mask sequence conditioned on an input image.<n>CoMoGen consistently outperforms prior controllable video generation methods and achieves state-of-the-art performance in motion fidelity and perceptual realism.
  arXiv  Detail & Related papers  (2026-05-21T19:51:20Z)
• OmniMotion-X: Versatile Multimodal Whole-Body Motion Generation [52.579531290307926]
  This paper introduces OmniMotion-X, a versatile framework for whole-body human motion generation.<n> OmniMotion-X efficiently supports diverse multimodal tasks, including text-to-motion, music-to-dance, speech-to-gesture.<n>To enable high-quality multimodal training, we construct OmniMoCap-X, the largest unified multimodal motion dataset to date.
  arXiv  Detail & Related papers  (2025-10-22T17:25:33Z)
• MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing [53.98607267063729]
  MotionVerse is a framework to comprehend, generate, and edit human motion in both single-person and multi-person scenarios.<n>We employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens.<n>We also introduce a textitDelay Parallel Modeling strategy, which temporally staggers the encoding of residual token streams.
  arXiv  Detail & Related papers  (2025-09-28T04:20:56Z)
• GenM$^3$: Generative Pretrained Multi-path Motion Model for Text Conditional Human Motion Generation [19.2804620329011]
  Generative Pretrained Multi-path Motion Model (GenM(3)) is a comprehensive framework designed to learn unified motion representations.<n>To enable large-scale training, we integrate and unify 11 high-quality motion datasets.<n>GenM(3) achieves a state-of-the-art FID of 0.035 on the HumanML3D benchmark, surpassing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2025-03-19T05:56:52Z)
• Large Motion Model for Unified Multi-Modal Motion Generation [50.56268006354396]
  Large Motion Model (LMM) is a motion-centric, multi-modal framework that unifies mainstream motion generation tasks into a generalist model. LMM tackles these challenges from three principled aspects.
  arXiv  Detail & Related papers  (2024-04-01T17:55:11Z)
• MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models [22.044020889631188]
  We introduce MambaTalk, enhancing gesture diversity and rhythm through multimodal integration.<n>Our method matches or exceeds the performance of state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-14T15:10:54Z)
• MoDi: Unconditional Motion Synthesis from Diverse Data [51.676055380546494]
  We present MoDi, an unconditional generative model that synthesizes diverse motions. Our model is trained in a completely unsupervised setting from a diverse, unstructured and unlabeled motion dataset. We show that despite the lack of any structure in the dataset, the latent space can be semantically clustered.
  arXiv  Detail & Related papers  (2022-06-16T09:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.