Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control

• URL: http://arxiv.org/abs/2607.03964v1
• Date: Sat, 04 Jul 2026 17:45:38 GMT
• Title: Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control
• Abstract summary: We introduce Worldscape-MoE, a Mixture-of-Experts world model for scalable heterogeneous action control.<n>Our key observation is that different controls specify different interfaces to the same underlying world.<n>Worldscape-MoE operationalizes this observation through modality-aware control injection, shared and control-specific experts, and a progressive MoE tuning strategy.
• Score: 31.020641426071293
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models are rapidly becoming a core infrastructure for embodied intelligence and interactive agents: they provide controllable simulators in which agents can perceive, act, forecast, and acquire scalable experience. Yet current video generation world models are still organized around isolated control interfaces, such as camera trajectories, robot actions, or hand-joint signals. This fragmentation is increasingly a scaling bottleneck. The central challenge is not the absence of controllable generators, but the lack of a unified and extensible learning framework that can absorb heterogeneous action supervision while preserving a shared model of world dynamics. In this work, we introduce Worldscape-MoE, a Mixture-of-Experts world model built on Diffusion Transformers for scalable heterogeneous action control. Our key observation is that different controls specify different interfaces to the same underlying world: although their representations differ, they constrain shared physical regularities, scene dynamics, and interaction semantics. Worldscape-MoE operationalizes this observation through modality-aware control injection, shared and control-specific experts, and a progressive MoE tuning strategy that supports continual extension to new action modalities. Experiments across locomotion, robotic manipulation, and egocentric hand control show that heterogeneous supervision improves rather than interferes with individual control capabilities. Worldscape-MoE achieves strong results on WorldArena, improves locomotion and hand-control metrics, exhibits robust out-of-distribution generalization, and demonstrates scaling behavior as additional control data and experts are integrated.

Related papers

• AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization [88.25615187835321]
  We propose AnyWorld, a cross-embodiment world modeling framework.<n>Our model factorizes an interaction into action, camera, and embodiment.<n>We train the model with large-scale human interaction pretraining.
  arXiv  Detail & Related papers  (2026-08-29T12:55:15Z)
• BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generation [22.803470671269874]
  We introduce BehaviorWorldGen, a framework that closes the loop between action models and world simulators.<n>Its core component is BehaviorFlow, a meta-action-conditioned traffic-flow model.<n>The resulting rollouts are rendered by a world simulator into realistic multi-view observations.
  arXiv  Detail & Related papers  (2026-08-23T02:53:09Z)
• ABot-M0.5: Unified Mobility-and-Manipulation World Action Model [27.262271208923995]
  ABot-M0.5 is built on the insight that mobile manipulation requires alignment at three levels: temporal granularity, action space, and train-test consistency.<n> Experiments on challenging mobile and fine-grained manipulation benchmarks demonstrate that ABot-M0.5 achieves state-of-the-art performance in both long-horizon task success and finegrained control accuracy.
  arXiv  Detail & Related papers  (2026-07-01T09:21:20Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• OMG: Omni-Modal Motion Generation for Generalist Humanoid Control [30.236954254902614]
  We argue that the key to general-purpose humanoid control is to build a scalable brain, atop a reactive motion tracking system.<n>We present OMG, which addresses two challenges: acquiring a vast amount of high-quality data to achieve general purpose control, and equipping the generator with the capability to condition on compositional, multi-modal inputs.
  arXiv  Detail & Related papers  (2026-06-09T02:46:14Z)
• DisCo: World Models with Discrete Camera Motion Control [79.86256515640231]
  We propose DisCo, a controllable video world model that conditions generation on a compact set of discrete action primitives to improve action separability.<n>DisCo achieves significantly more reliable action following while preserving visual quality.
  arXiv  Detail & Related papers  (2026-06-06T03:50:45Z)
• AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization [79.26935895370191]
  AnchorWorld is a framework that advances egocentric simulation through enhanced interaction integrity.<n> auxiliary training supervision incorporates viewpoints decoupled from the agent's first-person sensorium.<n>We propose a simple yet effective mechanism for customizing self-evolving worlds.
  arXiv  Detail & Related papers  (2026-06-05T14:43:13Z)
• Astra: General Interactive World Model with Autoregressive Denoising [73.6594791733982]
  Astra is an interactive general world model that generates real-world futures for diverse scenarios.<n>We propose an autoregressive denoising architecture and use temporal causal attention to aggregate past observations.<n>Astra achieves interactive, consistent, and general long-term video prediction and supports various forms of interactions.
  arXiv  Detail & Related papers  (2025-12-09T18:59:57Z)
• Interactive Character Control with Auto-Regressive Motion Diffusion Models [18.727066177880708]
  We propose A-MDM (Auto-regressive Motion Diffusion Model) for real-time motion synthesis. Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on previous frame. We introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2023-06-01T07:48:34Z)
• Isolating and Leveraging Controllable and Noncontrollable Visual Dynamics in World Models [65.97707691164558]
  We present Iso-Dream, which improves the Dream-to-Control framework in two aspects. First, by optimizing inverse dynamics, we encourage world model to learn controllable and noncontrollable sources. Second, we optimize the behavior of the agent on the decoupled latent imaginations of the world model.
  arXiv  Detail & Related papers  (2022-05-27T08:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.