Horizon Imagination: Efficient On-Policy Rollout in Diffusion World Models

• URL: http://arxiv.org/abs/2602.08032v2
• Date: Tue, 17 Feb 2026 18:50:34 GMT
• Title: Horizon Imagination: Efficient On-Policy Rollout in Diffusion World Models
• Abstract summary: We study diffusion-based world models for reinforcement learning.<n>Current methods either require heavyweight models at inference or rely on highly sequential imagination.<n>We propose Horizon Imagination (HI), an on-policy imagination process for discrete policies that denoises multiple future observations in parallel.
• Score: 53.30870288484743
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study diffusion-based world models for reinforcement learning, which offer high generative fidelity but face critical efficiency challenges in control. Current methods either require heavyweight models at inference or rely on highly sequential imagination, both of which impose prohibitive computational costs. We propose Horizon Imagination (HI), an on-policy imagination process for discrete stochastic policies that denoises multiple future observations in parallel. HI incorporates a stabilization mechanism and a novel sampling schedule that decouples the denoising budget from the effective horizon over which denoising is applied while also supporting sub-frame budgets. Experiments on Atari 100K and Craftium show that our approach maintains control performance with a sub-frame budget of half the denoising steps and achieves superior generation quality under varied schedules. Code is available at https://github.com/leor-c/horizon-imagination.

Related papers

• Offline Reinforcement Learning with Universal Horizon Models [15.801661284310823]
  We introduce universal horizon models (UHM), a generalization of GHM that directly predicts future states under arbitrary horizons.<n>We propose a scalable value learning method that employs a winsorized horizon distribution to stabilize training by capping excessively large horizons.<n> Experimental results on 100 challenging OGBench tasks demonstrate that the proposed method outperforms competitive baselines.
  arXiv  Detail & Related papers  (2026-05-15T04:30:30Z)
• Sword: Style-Robust World Models as Simulators via Dynamic Latent Bootstrapping for VLA Policy Post-Training [19.013714390803052]
  One approach treats learned World Models as generative simulators, enabling policy optimization entirely within "imagination"<n>However, when deployed as simulators for specific environments such as the LIBERO benchmark, existing World Models often suffer from poor generalization and long-horizon error accumulation.<n>We propose Sword, a robust World Model framework, to mitigate these problems.
  arXiv  Detail & Related papers  (2026-05-08T05:54:33Z)
• World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning [18.658825757342544]
  Vision-language models (VLMs) have shown strong performance on static visual understanding.<n>They still struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion.<n>We propose World2VLM, a training framework that distills spatial imagination from a generative world model into a vision-language model.
  arXiv  Detail & Related papers  (2026-04-29T17:48:01Z)
• Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method [54.461213497603154]
  Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities.<n>Nuplan-Occ is the largest occupancy dataset to date, constructed from the widely used Nuplan benchmark.<n>We develop a unified framework that jointly synthesizes high-quality occupancy, multi-view videos, and LiDAR point clouds.
  arXiv  Detail & Related papers  (2025-10-27T03:52:45Z)
• BADiff: Bandwidth Adaptive Diffusion Model [55.10134744772338]
  Traditional diffusion models produce high-fidelity images by performing a fixed number of denoising steps, regardless of downstream transmission limitations.<n>In practical cloud-to-device scenarios, limited bandwidth often necessitates heavy compression, leading to loss of fine textures and wasted computation.<n>We introduce a joint end-to-end training strategy where the diffusion model is conditioned on a target quality level derived from the available bandwidth.
  arXiv  Detail & Related papers  (2025-10-24T11:50:03Z)
• Lightning Fast Caching-based Parallel Denoising Prediction for Accelerating Talking Head Generation [50.04968365065964]
  Diffusion-based talking head models generate high-quality, photorealistic videos but suffer from slow inference.<n>We introduce Lightning-fast Caching-based Parallel denoising prediction (LightningCP)<n>We also propose Decoupled Foreground Attention (DFA) to further accelerate attention computations.
  arXiv  Detail & Related papers  (2025-08-25T02:58:39Z)
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion [67.94300151774085]
  We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models.<n>It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs.
  arXiv  Detail & Related papers  (2025-06-09T17:59:55Z)
• SphereDiff: Tuning-free Omnidirectional Panoramic Image and Video Generation via Spherical Latent Representation [31.305851707485967]
  We introduce SphereDiff, a novel approach for seamless 360-degree panoramic image and video generation using state-of-the-art diffusion models without additional tuning.<n>We extend MultiDiffusion to spherical latent space and propose a spherical latent sampling method to enable direct use of pretrained diffusion models.<n>Our method outperforms existing approaches in generating 360-degree panoramic content while maintaining high fidelity, making it a robust solution for immersive AR/VR applications.
  arXiv  Detail & Related papers  (2025-04-19T19:59:11Z)
• One-Step Diffusion Model for Image Motion-Deblurring [85.76149042561507]
  We propose a one-step diffusion model for deblurring (OSDD), a novel framework that reduces the denoising process to a single step.<n>To tackle fidelity loss in diffusion models, we introduce an enhanced variational autoencoder (eVAE), which improves structural restoration.<n>Our method achieves strong performance on both full and no-reference metrics.
  arXiv  Detail & Related papers  (2025-03-09T09:39:57Z)
• Avoiding mode collapse in diffusion models fine-tuned with reinforcement learning [0.0]
  Fine-tuning foundation models via reinforcement learning (RL) has proven promising for aligning to downstream objectives. We exploit the hierarchical nature of diffusion models (DMs) and train them dynamically at each epoch with a tailored RL method. We show that models trained with HRF achieve better preservation of diversity in downstream tasks, thus enhancing the fine-tuning robustness and at uncompromising mean rewards.
  arXiv  Detail & Related papers  (2024-10-10T19:06:23Z)
• Receding Horizon Inverse Reinforcement Learning [16.63162838890697]
  Inverse reinforcement learning (IRL) seeks to infer a cost function that explains the underlying goals and preferences of expert demonstrations. This paper presents receding horizon inverse reinforcement learning (RHIRL), a new IRL algorithm for high-dimensional, noisy, continuous systems with black-box dynamic models.
  arXiv  Detail & Related papers  (2022-06-09T13:03:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.