Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models

• URL: http://arxiv.org/abs/2601.08955v1
• Date: Tue, 13 Jan 2026 19:49:58 GMT
• Title: Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models
• Authors: Youwei Liu, Jian Wang, Hanlin Wang, Beichen Guo, Wenjie Li,
• Abstract summary: We propose Imagine-then-Plan (textttITP), a unified framework for agent learning via lookahead imagination.<n>We introduce a novel adaptive lookahead mechanism by trading off the ultimate goal and task progress.<n>Our experiments show that textttITP significantly outperforms competitive baselines.
• Score: 13.90532093512575
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in world models have shown promise for modeling future dynamics of environmental states, enabling agents to reason and act without accessing real environments. Current methods mainly perform single-step or fixed-horizon rollouts, leaving their potential for complex task planning under-exploited. We propose Imagine-then-Plan (\texttt{ITP}), a unified framework for agent learning via lookahead imagination, where an agent's policy model interacts with the learned world model, yielding multi-step ``imagined'' trajectories. Since the imagination horizon may vary by tasks and stages, we introduce a novel adaptive lookahead mechanism by trading off the ultimate goal and task progress. The resulting imagined trajectories provide rich signals about future consequences, such as achieved progress and potential conflicts, which are fused with current observations, formulating a partially \textit{observable} and \textit{imaginable} Markov decision process to guide policy learning. We instantiate \texttt{ITP} with both training-free and reinforcement-trained variants. Extensive experiments across representative agent benchmarks demonstrate that \texttt{ITP} significantly outperforms competitive baselines. Further analyses validate that our adaptive lookahead largely enhances agents' reasoning capability, providing valuable insights into addressing broader, complex tasks.

Related papers

• Current Agents Fail to Leverage World Model as Tool for Foresight [61.82522354207919]
  Generative world models offer a promising remedy: agents could use them to foresee outcomes before acting.<n>This paper empirically examines whether current agents can leverage such world models as tools to enhance their cognition.
  arXiv  Detail & Related papers  (2026-01-07T13:15:23Z)
• PerspAct: Enhancing LLM Situated Collaboration Skills through Perspective Taking and Active Vision [2.32300953742759]
  This study evaluates whether explicitly incorporating diverse points of view using the ReAct framework can enhance an LLM's ability to understand and ground the demands of other agents.<n>We introduce active visual exploration across a suite of seven scenarios of increasing perspective-taking complexity.<n>Our results demonstrate that explicit perspective cues, combined with active exploration strategies, significantly improve the model's interpretative accuracy and collaborative effectiveness.
  arXiv  Detail & Related papers  (2025-11-11T10:54:15Z)
• Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challenges [87.02855999212817]
  Edge General Intelligence (EGI) represents a transformative evolution of edge computing, where distributed agents possess the capability to perceive, reason, and act autonomously.<n>World models act as proactive internal simulators that not only predict but also actively imagine future trajectories, reason under uncertainty, and plan multi-step actions with foresight.<n>This survey bridges the gap by offering a comprehensive analysis of how world models can empower agentic artificial intelligence (AI) systems at the edge.
  arXiv  Detail & Related papers  (2025-08-13T07:29:40Z)
• World Models for Cognitive Agents: Transforming Edge Intelligence in Future Networks [55.90051810762702]
  We present a comprehensive overview of world models, highlighting their architecture, training paradigms, and applications across prediction, generation, planning, and causal reasoning.<n>We propose Wireless Dreamer, a novel world model-based reinforcement learning framework tailored for wireless edge intelligence optimization.
  arXiv  Detail & Related papers  (2025-05-31T06:43:00Z)
• Deep Active Inference Agents for Delayed and Long-Horizon Environments [1.693200946453174]
  AIF agents rely on accurate immediate predictions and exhaustive planning, a limitation that is exacerbated in delayed environments.<n>We propose a generative-policy architecture featuring a multi-step latent transition that lets the generative model predict an entire horizon in a single look-ahead.<n>We evaluate our agent in an environment that mimics a realistic industrial scenario with delayed and long-horizon settings.
  arXiv  Detail & Related papers  (2025-05-26T11:50:22Z)
• On the Modeling Capabilities of Large Language Models for Sequential Decision Making [52.128546842746246]
  Large pretrained models are showing increasingly better performance in reasoning and planning tasks. We evaluate their ability to produce decision-making policies, either directly, by generating actions, or indirectly. In environments with unfamiliar dynamics, we explore how fine-tuning LLMs with synthetic data can significantly improve their reward modeling capabilities.
  arXiv  Detail & Related papers  (2024-10-08T03:12:57Z)
• Grounded Answers for Multi-agent Decision-making Problem through Generative World Model [27.263093790379024]
  generative models often produce sketchy and misleading solutions for complex multi-agent decision-making problems. We show a paradigm that integrates a language-guided simulator into the multi-agent reinforcement learning pipeline to enhance the generated answer. In particular, it can generate consistent interaction sequences and explainable reward functions at interaction states, opening the path for training generative models of the future.
  arXiv  Detail & Related papers  (2024-10-03T16:49:59Z)
• DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation [107.5934592892763]
  We propose DREAMWALKER -- a world model based VLN-CE agent. The world model is built to summarize the visual, topological, and dynamic properties of the complicated continuous environment. It can simulate and evaluate possible plans entirely in such internal abstract world, before executing costly actions.
  arXiv  Detail & Related papers  (2023-08-14T23:45:01Z)
• Continual Visual Reinforcement Learning with A Life-Long World Model [55.05017177980985]
  We present a new continual learning approach for visual dynamics modeling.<n>We first introduce the life-long world model, which learns task-specific latent dynamics.<n>Then, we address the value estimation challenge for previous tasks with the exploratory-conservative behavior learning approach.
  arXiv  Detail & Related papers  (2023-03-12T05:08:03Z)
• Investigating the role of model-based learning in exploration and transfer [11.652741003589027]
  In this paper, we investigate transfer learning in the context of model-based agents. We find that a model-based approach outperforms controlled model-free baselines for transfer learning. Our results show that intrinsic exploration combined with environment models present a viable direction towards agents that are self-supervised and able to generalize to novel reward functions.
  arXiv  Detail & Related papers  (2023-02-08T11:49:58Z)
• Goal-Aware Prediction: Learning to Model What Matters [105.43098326577434]
  One of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model and that of the downstream planner or policy. We propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space. We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.
  arXiv  Detail & Related papers  (2020-07-14T16:42:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.