OMG: Omni-Modal Motion Generation for Generalist Humanoid Control

• URL: http://arxiv.org/abs/2606.10340v1
• Date: Tue, 09 Jun 2026 02:46:14 GMT
• Title: OMG: Omni-Modal Motion Generation for Generalist Humanoid Control
• Abstract summary: We argue that the key to general-purpose humanoid control is to build a scalable brain, atop a reactive motion tracking system.<n>We present OMG, which addresses two challenges: acquiring a vast amount of high-quality data to achieve general purpose control, and equipping the generator with the capability to condition on compositional, multi-modal inputs.
• Score: 30.236954254902614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humanoid whole-body control has made significant progress in recent years, yet existing approaches remain limited to few-skill policies with heavy reward engineering, or motion trackers that are difficult to extend to new input modalities. We argue that the key to general-purpose humanoid control is to build a scalable brain, a module capable of reasoning with diverse conditioning modalities, atop a reactive motion tracking cerebellum, mirroring the hierarchical structure of biological motor systems. Two challenges arise in realizing this vision: acquiring a vast amount of high-quality data to achieve general purpose control, and equipping the generator with the capability to condition on compositional, extensible multi-modal inputs. We present OMG, which addresses these challenges with a meticulous data curation, filtering and labeling pipeline, as well as a diffusion-based motion generation backbone that conditions on language, audio, and human reference motions. Extensive experiments validate OMG as an omni-modal whole-body controller exhibiting state-of-the-art performance, model scaling behavior and efficient adaptation to new distributions and modalities, marking a concrete step toward foundation models for humanoid robots.

Related papers

• Scaling Behavior Foundation Model for Humanoid Robots [77.31115920959364]
  Behavior Foundation Models (BFMs) have emerged as a promising solution to address challenges by leveraging large-scale behavioral data.<n>We show that our approach yields significant improvements in control fidelity and task generalization.<n>These results establish BFM as a principled and effective foundation for scalable and general-purpose humanoid control.
  arXiv  Detail & Related papers  (2026-07-16T16:08:27Z)
• Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control [31.020641426071293]
  We introduce Worldscape-MoE, a Mixture-of-Experts world model for scalable heterogeneous action control.<n>Our key observation is that different controls specify different interfaces to the same underlying world.<n>Worldscape-MoE operationalizes this observation through modality-aware control injection, shared and control-specific experts, and a progressive MoE tuning strategy.
  arXiv  Detail & Related papers  (2026-07-04T17:45:38Z)
• M3imic: Learning a Versatile Whole-Body Controller for Multimodal Motion Mimicking [45.636871702422134]
  Multi-Modal Mimic is a versatile whole-body control framework for humanoid robots.<n>It unifies heterogeneous motion reference modalities, including robot joint angles, human pose trajectories, and end-effector poses.<n>It achieves sim-to-real transfer across multiple motion reference modalities without modality-specific retraining.
  arXiv  Detail & Related papers  (2026-06-03T12:52:37Z)
• AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling [60.13338774249153]
  We introduce OmniHuMo, a dataset of over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations.<n>We propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer.<n>Experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.
  arXiv  Detail & Related papers  (2026-05-28T07:15:19Z)
• ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation [66.02142169323521]
  Vision-Language-Action models and world models have recently emerged as promising paradigms for general-purpose robotic intelligence.<n>Existing benchmarks are largely simulator-centric, which provide controllability but fail to capture the reality gap caused by perception noise.<n>We introduce ManipArena, a standardized evaluation framework designed to bridge simulation and real-world execution.
  arXiv  Detail & Related papers  (2026-03-30T15:06:41Z)
• ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation [55.467742403416175]
  We introduce a physics-driven neural algorithm that translates large-scale motion capture to humanoid embodiments.<n>We learn a unified multimodal controller that supports both dense references and sparse task specifications.<n>Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception.
  arXiv  Detail & Related papers  (2026-03-03T18:59:29Z)
• PMG: Parameterized Motion Generator for Human-like Locomotion Control [14.637220434597168]
  We develop a real-time motion generator that produces human-like locomotion in a single integrated system.<n>We show that, within a single integrated system, PMG produces natural, human-like locomotion, responds precisely to high-dimensional control inputs.<n>These results establish a practical, experimentally validated pathway toward natural and deployable humanoid control.
  arXiv  Detail & Related papers  (2026-02-13T06:38:04Z)
• OWMM-Agent: Open World Mobile Manipulation With Multi-modal Agentic Data Synthesis [70.39500621448383]
  Open-world mobile manipulation task remains a challenge due to the need for generalization to open-ended instructions and environments.<n>We propose a novel multi-modal agent architecture that maintains multi-view scene frames and agent states for decision-making and controls the robot by function calling.<n>We highlight our fine-tuned OWMM-VLM as the first dedicated foundation model for mobile manipulators with global scene understanding, robot state tracking, and multi-modal action generation in a unified model.
  arXiv  Detail & Related papers  (2025-06-04T17:57:44Z)
• GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control [122.65089441381741]
  We present GEM, a Generalizable Ego-vision Multimodal world model.<n>It predicts future frames using a reference frame, sparse features, human poses, and ego-trajectories.<n>Our dataset is comprised of 4000+ hours of multimodal data across domains like autonomous driving, egocentric human activities, and drone flights.
  arXiv  Detail & Related papers  (2024-12-15T14:21:19Z)
• Interactive Character Control with Auto-Regressive Motion Diffusion Models [18.727066177880708]
  We propose A-MDM (Auto-regressive Motion Diffusion Model) for real-time motion synthesis. Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on previous frame. We introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2023-06-01T07:48:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.