WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling

• URL: http://arxiv.org/abs/2607.03461v1
• Date: Fri, 03 Jul 2026 16:18:20 GMT
• Title: WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling
• Abstract summary: World models aim to capture environment dynamics in ways that support perception, reasoning, and action.<n>We introduce textttWorldBagel, a unified VLAW framework built on BAGEL, a modern multimodal unified model.<n>We show that textttWorldBagel consistently outperforms task-specific alternatives and learns action representations that are more structured and semantically aligned with visual and linguistic context.
• Score: 53.14409745466386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in Vision-Language-Action-World (VLAW) modeling. Meanwhile, unified vision-language models have demonstrated strong multimodal generation capabilities, yet their potential as world models remains underexplored. In this work, we introduce \texttt{WorldBagel}, a unified VLAW framework built on BAGEL, a modern multimodal unified model, and use it to systematically investigate the role of unification in world modeling. Across multi-task robotic manipulation and cross-domain experiments, \texttt{WorldBagel} consistently outperforms task-specific alternatives and learns action representations that are more structured and semantically aligned with visual and linguistic context. Experiments on LIBERO, Language Table, and Franka show that unification is not only an architectural convenience, but also a key factor in learning effective VLAW models, leading to consistent empirical gains and deeper insights into multimodal world modeling. Code and model checkpoints will be released upon acceptance.

Related papers

• World Action Models: The Next Frontier in Embodied AI [123.5787299299832]
  Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning.<n>They learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention.<n>A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline.
  arXiv  Detail & Related papers  (2026-05-12T13:10:52Z)
• MotuBrain: An Advanced World Action Model for Robot Control [23.733029557644354]
  We present MotuBrain, a unified World Action Model that jointly models video and action under a UniDiffuser formulation.<n>A single model supports policy learning, world modeling, video generation, inverse dynamics, and joint video-action prediction.<n>Building on Motus, MotuBrain further introduces unified multiview modeling, an independent text stream for stronger language-action coupling.<n>Our inference stack combines step reduction, compilation, FP8 quantization, DiT caching, V2A-style action-only inference, and real-time chunked closed-loop execution.
  arXiv  Detail & Related papers  (2026-04-30T12:34:44Z)
• Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models [35.944951371813296]
  Vision-Language-Action (VLA) models show strong generalization for robotic control, but finetuning them with reinforcement learning (RL) is constrained by the high cost and safety risks of real-world interaction.<n>We propose VLA-MBPO, a practical framework to tackle these problems in VLA finetuning.<n>Our approach has three key design choices: (i) adapting unified multimodal models for data-efficient world modeling; (ii) an interleaved view decoding mechanism to enforce multi-view consistency; and (iii) chunk-level branched rollout to mitigate error compounding.
  arXiv  Detail & Related papers  (2026-03-21T02:44:39Z)
• Beyond Language Modeling: An Exploration of Multimodal Pretraining [125.34714978184638]
  We provide empirical clarity through controlled, from-scratch pretraining experiments.<n>We adopt the Transfusion framework, using next-token prediction for language and diffusion for vision.<n>We demonstrate that the MoE architecture harmonizes this scaling asymmetry by providing the high model capacity required by language.
  arXiv  Detail & Related papers  (2026-03-03T18:58:00Z)
• From Word to World: Can Large Language Models be Implicit Text-based World Models? [82.47317196099907]
  Agentic reinforcement learning increasingly relies on experience-driven scaling.<n>World models offer a potential way to improve learning efficiency through simulated experience.<n>We study whether large language models can reliably serve this role and under what conditions they meaningfully benefit agents.
  arXiv  Detail & Related papers  (2025-12-21T17:28:42Z)
• Co-Evolving Latent Action World Models [57.48921576959243]
  Adapting pre-trained video models into controllable world models via latent actions is a promising step towards creating generalist world models.<n>We propose CoLA-World, which for the first time successfully realizes this synergistic paradigm.<n>This unlocks a co-evolution cycle: the world model acts as a knowledgeable tutor, providing gradients to shape a high-quality LAM.
  arXiv  Detail & Related papers  (2025-10-30T12:28:40Z)
• Can World Models Benefit VLMs for World Dynamics? [59.73433292793044]
  We investigate the capabilities when world model priors are transferred into Vision-Language Models.<n>We name our best-performing variant Dynamic Vision Aligner (DyVA)<n>We find DyVA to surpass both open-source and proprietary baselines, achieving state-of-the-art or comparable performance.
  arXiv  Detail & Related papers  (2025-10-01T13:07:05Z)
• OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving [12.004183122121042]
  OccLLaMA is an occupancy-language-action generative world model. We build a unified multi-modal vocabulary for vision, language and action. OccLLaMA achieves competitive performance across multiple tasks.
  arXiv  Detail & Related papers  (2024-09-05T06:30:01Z)
• WorldGPT: Empowering LLM as Multimodal World Model [51.243464216500975]
  We introduce WorldGPT, a generalist world model built upon Multimodal Large Language Model (MLLM) WorldGPT acquires an understanding of world dynamics through analyzing millions of videos across various domains. We conduct evaluations on WorldNet, a multimodal state transition prediction benchmark.
  arXiv  Detail & Related papers  (2024-04-28T14:42:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.