Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training

• URL: http://arxiv.org/abs/2604.21741v1
• Date: Thu, 23 Apr 2026 14:42:54 GMT
• Title: Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training
• Abstract summary: We propose a post-training framework that uses a learned world model as a reusable corrective substrate for failure-targeted policy improvement.<n>Hi-WM caches intermediate states and supports rollback and branching, allowing a single failure state to be reused for multiple corrective continuations.<n>We evaluate Hi-WM on three real-world manipulation tasks spanning both rigid and deformable object interaction, and on two policy backbones.
• Score: 54.896907620476675
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Post-training is essential for turning pretrained generalist robot policies into reliable task-specific controllers, but existing human-in-the-loop pipelines remain tied to physical execution: each correction requires robot time, scene setup, resets, and operator supervision in the real world. Meanwhile, action-conditioned world models have been studied mainly for imagination, synthetic data generation, and policy evaluation. We propose \textbf{Human-in-the-World-Model (Hi-WM)}, a post-training framework that uses a learned world model as a reusable corrective substrate for failure-targeted policy improvement. A policy is first rolled out in closed loop inside the world model; when the rollout becomes incorrect or failure-prone, a human intervenes directly in the model to provide short corrective actions. Hi-WM caches intermediate states and supports rollback and branching, allowing a single failure state to be reused for multiple corrective continuations and yielding dense supervision around behaviors that the base policy handles poorly. The resulting corrective trajectories are then added back to the training set for post-training. We evaluate Hi-WM on three real-world manipulation tasks spanning both rigid and deformable object interaction, and on two policy backbones. Hi-WM improves real-world success by 37.9 points on average over the base policy and by 19.0 points over a world-model closed-loop baseline, while world-model evaluation correlates strongly with real-world performance (r = 0.953). These results suggest that world models can serve not only as generators or evaluators, but also as effective corrective substrates for scalable robot post-training.

Related papers

• REVOLVE: An Automated Closed-Loop Framework for Evolving Robot Manipulation with Minimal Human Intervention [17.399763167916195]
  We present REVOLVE, an automated closed-loop framework for evolving robot manipulation with minimal human intervention.<n>Built on a unified software platform, REVOLVE integrates data collection, policy training and deployment, failure recovery, and continual learning into a single closed-loop workflow.
  arXiv  Detail & Related papers  (2026-09-13T16:08:03Z)
• HaWMPO: Hallucination-Aware World Model-based Policy Optimization for Generalist Robot Policy [22.534022126501373]
  Generalist robot policies have demonstrated strong generalization across robotic manipulation tasks.<n>Recent methods improve Visual-Language-Action (VLA) policies through online reinforcement learning on real robots.<n>World models offer a promising alternative by enabling policy optimization with imagined rollouts.<n>We propose HaWMPO, a closed-loop reinforcement learning pipeline for VLA policy post-training with world models.
  arXiv  Detail & Related papers  (2026-09-09T09:33:25Z)
• Q-Learning With World Models [69.63968749241239]
  We propose a framework that leverages world models to perform test-time search over imagined trajectories on top of Q-learning to select high-value actions.<n>QWM significantly outperforms strong prior state-of-the-art methods on both sample efficiency and performance.
  arXiv  Detail & Related papers  (2026-08-17T22:00:42Z)
• GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation [73.02743604295246]
  WMBench is a benchmark constructed from real-robot teleoperation data and matched policy rollouts.<n>We analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions.<n>Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism.
  arXiv  Detail & Related papers  (2026-07-02T17:02:43Z)
• RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation [52.855830378221775]
  Video world models are emerging as a scalable alternative for evaluating generalist robot policies.<n>We introduce RoboWorld, an automated evaluation pipeline that pairs a fast video world model with a task-progress-aware vision-language model scoring.
  arXiv  Detail & Related papers  (2026-07-01T15:22:41Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Ctrl-World: A Controllable Generative World Model for Robot Manipulation [53.71061464925014]
  Generalist robot policies can perform a wide range of manipulation skills.<n> evaluating and improving their ability with unfamiliar objects and instructions remains a significant challenge.<n>World models offer a promising, scalable alternative by enabling policies to rollout within imagination space.
  arXiv  Detail & Related papers  (2025-10-11T09:13:10Z)
• WorldGym: World Model as An Environment for Policy Evaluation [41.204900701616914]
  WorldGym is an autoregressive, action-conditioned video generation model which serves as a proxy to real world environments.<n> Policies are evaluated via Monte Carlo rollouts in the world model, with a vision-language model providing rewards.<n>We show that WorldGym is able to preserve relative policy rankings across different policy versions, sizes, and training checkpoints.
  arXiv  Detail & Related papers  (2025-05-31T15:51:56Z)
• Offline Robotic World Model: Learning Robotic Policies without a Physics Simulator [50.191655141020505]
  Reinforcement Learning (RL) has demonstrated impressive capabilities in robotic control but remains challenging due to high sample complexity, safety concerns, and the sim-to-real gap.<n>We introduce Offline Robotic World Model (RWM-O), a model-based approach that explicitly estimates uncertainty to improve policy learning without reliance on a physics simulator.
  arXiv  Detail & Related papers  (2025-04-23T12:58:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.