Grounded Iterative Language Planning: How Parameterized World Models Reduce Hallucination Propagation in LLM Agents

• URL: http://arxiv.org/abs/2606.27806v2
• Date: Wed, 01 Jul 2026 18:42:32 GMT
• Title: Grounded Iterative Language Planning: How Parameterized World Models Reduce Hallucination Propagation in LLM Agents
• Abstract summary: An agent-based world model calls an API and reasons flexibly in language, but its errors appear as hallucinated state changes that are hard to score with ordinary regression losses.<n>A parameterized world model is a trained transition predictor; its errors are easier to measure with quantities such as NodeMSE, delta accuracy, and validity accuracy.<n>We compare these two families on four graph-structured planning benchmarks and introduce operational hallucination metrics for the agent-based case.<n>The comparison motivates Grounded Iterative Language Planning(GILP), which trains only a small parameterized backbone and combines it with API-based agent reasoning.
• Score: 2.571318992830046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models for language agents come in two useful forms. An agent-based world model calls an LLM API and reasons flexibly in language, but its errors appear as hallucinated state changes that are hard to score with ordinary regression losses. A parameterized world model is a trained transition predictor; its errors are easier to measure with quantities such as NodeMSE, delta accuracy, and validity accuracy, but it is usually weaker as a standalone planner. We compare these two families on four graph-structured planning benchmarks and introduce operational hallucination metrics for the agent-based case. The comparison motivates Grounded Iterative Language Planning(GILP), which trains only a small parameterized backbone and combines it with API-based agent reasoning. The backbone supplies valid actions, predicted state deltas, risk, and value; the LLM drafts an action and imagined delta; and a consistency gate asks for revision when the two disagree. On real GPT-4o-mini calls, GILP reduces hallucinated-state rate from 0.176 to 0.035. In calibrated simulator ablations, it raises success from 0.668 to 0.838 while adding only ~22% extra LLM calls.

Related papers

• Towards Robust Reinforcement Learning for Small-Scale Language Model Agents [5.301609879131692]
  In this paper, fifteen (model, corpus) configurations were trained using Proximal Policy Optimization (PPO)<n>The experiments included Pythia-70M, 160M, 410M and SmolLM2-135M, 360M on the TinyStories, CNN/DailyMail, and Wiki-text103 corpora.<n>The proposed system converged stably in all experiments and improved preference win rate over the SFT baseline in configurations with a fluent prior and an informative reward signal.
  arXiv  Detail & Related papers  (2026-07-27T21:30:44Z)
• Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures [0.0]
  Causal Agent Replay (CAR) models an agent run as a structural causal model.<n>It applies a do-operation to a step, and re-executes a trajectory forward under the same policy.<n>CAR is open source and runs on hosted or free local models.
  arXiv  Detail & Related papers  (2026-06-06T17:44:23Z)
• Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation [4.089259624354187]
  Small open-source code models that power autocomplete still emit Fill-in-the-Middle (FIM) completions.<n>We propose an execution-free alternative: use frontier code models to synthesise plausible-but-wrong completions as hard negatives.
  arXiv  Detail & Related papers  (2026-06-02T04:15:50Z)
• ReDAct: Uncertainty-Aware Deferral for LLM Agents [61.507376922278894]
  We propose ReDAct (Reason-Defer-Act) to solve sequential decision-making problems.<n>In ReDAct, an agent is equipped with two LLMs: a small, cheap model used by default, and a large, more reliable but expensive model.<n>We show that deferring only about 15% of decisions to the large model can match the quality of using it exclusively, while significantly reducing inference costs.
  arXiv  Detail & Related papers  (2026-04-08T12:51:01Z)
• Tiny-Critic RAG: Empowering Agentic Fallback with Parameter-Efficient Small Language Models [7.704706624419061]
  Retrieval-Augmented Generation grounds Large Language Models (LLMs) to mitigate factual hallucinations.<n>We propose Tiny-Critic RAG, a parameter-efficient Small Language Model (SLM) via Low-Rank Adaptation (LoRA)<n>Tiny-Critic RAG achieves routing accuracy comparable to GPT-4o-mini while reducing latency by an order of magnitude.
  arXiv  Detail & Related papers  (2026-03-01T00:16:31Z)
• MERGETUNE: Continued fine-tuning of vision-language models [77.8627788911249]
  Fine-tuning vision-language models (VLMs) often leads to catastrophic forgetting of pretrained knowledge.<n>We introduce a novel paradigm, continued fine-tuning (CFT), which seeks to recover pretrained knowledge after a zero-shot model has already been adapted.
  arXiv  Detail & Related papers  (2026-01-15T15:15:53Z)
• Plan Verification for LLM-Based Embodied Task Completion Agents [10.439882851477162]
  Large language model (LLM) based task plans and corresponding human demonstrations for embodied AI may be noisy.<n>We propose an iterative verification framework in which a Judge LLM critiques action sequences and a Planner LLM applies the revisions.
  arXiv  Detail & Related papers  (2025-09-02T19:06:56Z)
• Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks [17.067788440109137]
  Mixture-of-Experts (MoE) models are now standard in state-of-the-art systems.<n>We investigate how MoE sparsity influences two distinct capability regimes: memorization skills and reasoning skills.
  arXiv  Detail & Related papers  (2025-08-26T04:31:28Z)
• Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [68.29746557968107]
  We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans.<n> Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
  arXiv  Detail & Related papers  (2024-05-23T08:33:19Z)
• Fine-tuning Language Models for Factuality [96.5203774943198]
  Large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines. Yet language models are prone to making convincing but factually inaccurate claims, often referred to as 'hallucinations' In this work, we fine-tune language models to be more factual, without human labeling.
  arXiv  Detail & Related papers  (2023-11-14T18:59:15Z)
• Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling [101.59430768507997]
  Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of the world. We propose using few-shot large language models (LLMs) to hypothesize an Abstract World Model (AWM) Our method of hypothesizing an AWM with LLMs and then verifying the AWM based on agent experience not only increases sample efficiency over contemporary methods by an order of magnitude.
  arXiv  Detail & Related papers  (2023-01-28T02:04:07Z)
• Mismatched No More: Joint Model-Policy Optimization for Model-Based RL [172.37829823752364]
  We propose a single objective for jointly training the model and the policy, such that updates to either component increases a lower bound on expected return. Our objective is a global lower bound on expected return, and this bound becomes tight under certain assumptions. The resulting algorithm (MnM) is conceptually similar to a GAN.
  arXiv  Detail & Related papers  (2021-10-06T13:43:27Z)
• LoRA: Low-Rank Adaptation of Large Language Models [71.75808607987281]
  Low-Rank Adaptation, or LoRA, freezes the pre-trained model weights and injects trainable rank decomposition into each layer of the Transformer architecture. For GPT-3, LoRA can reduce the number of trainable parameters by 10,000 times and the computation hardware requirement by 3 times compared to full fine-tuning.
  arXiv  Detail & Related papers  (2021-06-17T17:37:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.