Unifying Object-Centric World Models and Diffusion Policy: A Hierarchical Framework for Multi-Stage Robotic Tasks

• URL: http://arxiv.org/abs/2606.08775v1
• Date: Sun, 07 Jun 2026 18:39:45 GMT
• Title: Unifying Object-Centric World Models and Diffusion Policy: A Hierarchical Framework for Multi-Stage Robotic Tasks
• Abstract summary: We introduce WorldDP, a world model framework designed for multi-stage robotic manipulation.<n>Our hierarchical approach utilizes a high-level world model as a transition function to optimize for feasible subgoals during runtime.<n>We incorporate object-centric representations that decouple environmental entities and enable us to plan sequentially with respect to each.
• Score: 44.06996146648851
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual world models have shown great potential in learning complex system dynamics. Recent advancements leverage these models as transition functions within Model Predictive Control (MPC) frameworks to solve various control tasks. When applied to robotics, however, they are limited to single-stage tasks such as reaching or grasping, and struggle with multi-stage ones that demand complex sequential planning. In this work, we introduce WorldDP, a world model framework designed for multi-stage robotic manipulation. Our hierarchical approach utilizes a high-level world model as a transition function to optimize for feasible subgoals during runtime, which are subsequently reached by a low-level Diffusion Policy. To further aid in learning dynamics and planning, we incorporate object-centric representations that decouple environmental entities and enable us to plan sequentially with respect to each. Evaluated across several robotics benchmarks, WorldDP consistently outperforms existing baselines, validating that coupling the world model's physically grounded planning with diffusion policy's efficient execution yields superior multi-stage performance.

Related papers

• World Action Planner: Generalizable Decision-Making with Action-Conditioned World Models [54.68021681438324]
  We propose World Action Planner, a robot planning system that leverages the reasoning capabilities of Vision-Language Models (VLMs)<n>Our system enables an agent to propose initial action plans and iteratively refine them via optimization and search, reasoning over imagined world model rollouts.<n>We demonstrate that our approach achieves superior performance across compositional tasks, new layouts, and zero-shot generalization scenarios.
  arXiv  Detail & Related papers  (2026-07-30T02:41:52Z)
• Hierarchical Planning with Latent World Models [49.82095442483551]
  hierarchical planning achieves higher success while requiring up to 4x less planning-time compute.<n>We demonstrate that this hierarchical approach enables zero-shot control on real-world non-greedy robotic tasks.
  arXiv  Detail & Related papers  (2026-04-03T17:32:36Z)
• Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models [35.944951371813296]
  Vision-Language-Action (VLA) models show strong generalization for robotic control, but finetuning them with reinforcement learning (RL) is constrained by the high cost and safety risks of real-world interaction.<n>We propose VLA-MBPO, a practical framework to tackle these problems in VLA finetuning.<n>Our approach has three key design choices: (i) adapting unified multimodal models for data-efficient world modeling; (ii) an interleaved view decoding mechanism to enforce multi-view consistency; and (iii) chunk-level branched rollout to mitigate error compounding.
  arXiv  Detail & Related papers  (2026-03-21T02:44:39Z)
• From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction [57.56072009935036]
  We introduce a new driving paradigm named Policy World Model (PWM)<n>PWM integrates world modeling and trajectory planning within a unified architecture.<n>Our method matches or exceeds state-of-the-art approaches that rely on multi-view and multi-modal inputs.
  arXiv  Detail & Related papers  (2025-10-22T14:57:51Z)
• ExoPredicator: Learning Abstract Models of Dynamic Worlds for Robot Planning [77.49815848173613]
  We propose a framework for abstract world models that jointly learns symbolic state representations and causal processes for both endogenous actions and mechanisms.<n>Across five simulated tabletop robotics environments, the learned models enable fast planning that generalizes to held-out tasks with more objects and more complex goals, outperforming a range of baselines.
  arXiv  Detail & Related papers  (2025-09-30T13:44:34Z)
• Generative World Models of Tasks: LLM-Driven Hierarchical Scaffolding for Embodied Agents [0.0]
  We propose an effective world model for decision-making that models the world's physics and its task semantics.<n>A systematic review of 2024 research in low-resource multi-agent soccer reveals a clear trend towards integrating symbolic and hierarchical methods.<n>We formalize this trend into a framework for Hierarchical Task Environments (HTEs), which are essential for bridging the gap between simple, reactive behaviors and sophisticated, strategic team play.
  arXiv  Detail & Related papers  (2025-09-05T01:03:51Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• VDFD: Multi-Agent Value Decomposition Framework with Disentangled World Model [10.36125908359289]
  We propose a novel model-based multi-agent reinforcement learning approach named Value Decomposition Framework with Disentangled World Model.<n>Our method achieves high sample efficiency and exhibits superior performance compared to other baselines across a wide range of multi-agent learning tasks.
  arXiv  Detail & Related papers  (2023-09-08T22:12:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.