The Context Gathering Decision Process: A POMDP Framework for Agentic Search

• URL: http://arxiv.org/abs/2605.07042v1
• Date: Thu, 07 May 2026 23:45:07 GMT
• Title: The Context Gathering Decision Process: A POMDP Framework for Agentic Search
• Abstract summary: Large Language Model (LLM) agents are deployed in complex environments.<n>Without explicit infrastructure, an agent's working memory can degrade into lossy representations of the search state.<n>We formalize this challenge as the Context Gathering Decision Process (CGDP)<n>We derive two plug-and-play interventions for iterative LLM agents.
• Score: 38.92972416925679
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large Language Model (LLM) agents are deployed in complex environments -- such as massive codebases, enterprise databases, and conversational histories -- where the relevant state far exceeds their context windows. To navigate these spaces, an agent must iteratively explore the environment to find relevant information. However, without explicit infrastructure, an agent's working memory can degrade into lossy representations of the search state, resulting in redundant work (e.g. repetitive looping) and premature stopping. In this work, we formalize this challenge as the Context Gathering Decision Process (CGDP), a specialized Partially Observable Markov Decision Process, where an agent's objective is to adaptively refine its belief state to isolate the necessary information for a task. We model an LLM's behavior as approximate Thompson Sampling within this CGDP, and introduce a predicate-based method that decomposes an LLM's implicit search into explicit and modular operations. We then derive two plug-and-play interventions for iterative LLM agents: a persistent, predicate-based belief state that bounds context while preserving multi-hop reasoning, and a programmatic exhaustion gate that halts unproductive search without premature stopping. Across four methods and three question-answering domains, we empirically validate that replacing an LLM's implicit state with our CGDP-motivated belief state improves multi-hop reasoning by up to $11.4\%$; while the modular programmatic exhaustion detection saves up to $39\%$ of tokens without any degradation in agent performance. Ultimately, we argue that framing the LLM agent loop as a CGDP can guide the design of modular, non-interfering improvements to agentic search harnesses.

Related papers

• The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows [72.9051182568928]
  We present ReASearch, a unified framework for reasoning-driven optimization.<n>With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML.<n>We observe that complex search behaviors emerge naturally from the agent's reasoning process.
  arXiv  Detail & Related papers  (2026-08-07T02:15:39Z)
• Trajectory-Aware Retrieval Agents for Temporal Decision- Making [5.789945004686575]
  We study the problem of decision-making from long-form, temporally structured text using large language model (LLM) agents.<n>We introduce TLM, a closed-loop agentic framework that iteratively refines the evidence set using SHAP-guided feedback.<n> Empirically, TLM is evaluated on three temporally grounded decision tasks: medical question answering, earnings call surprise prediction, and overnight stock gap prediction.
  arXiv  Detail & Related papers  (2026-07-07T16:28:38Z)
• FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments [60.3427704389541]
  Large Language Models are being increasingly deployed as the decision-making core of autonomous agents.<n>Yet, in conversational benchmarks, these agents frequently fail due to the cascading effects of incorrect decision-making.<n>We present the Failure-Aware Meta-Agentic framework to tackle these challenges.
  arXiv  Detail & Related papers  (2026-04-28T02:21:53Z)
• Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search [56.78490647843876]
  Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use.<n>We propose bfM-ASK, a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context.
  arXiv  Detail & Related papers  (2026-01-08T08:13:27Z)
• Learning to Refine: An Agentic RL Approach for Iterative SPARQL Query Construction [0.18907108368038208]
  Current methods lack the adaptive policies needed to dynamically debug queries based on real-time execution feedback.<n>This paper introduces a novel agentic framework where an LLM learns a resilient policy for the sequential process of iterative SPARQL construction.<n>We show that a compact 3B- parameter model, trained exclusively via outcome-driven Reinforcement Learning (GRPO), can learn effective policies for this task.
  arXiv  Detail & Related papers  (2025-11-14T08:44:58Z)
• AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress [71.02263260394261]
  Large language models (LLMs) still encounter challenges in multi-turn decision-making tasks.<n>We build process reward models (PRMs) to evaluate each decision and guide the agent's decision-making process.<n>AgentPRM captures both the interdependence between sequential decisions and their contribution to the final goal.
  arXiv  Detail & Related papers  (2025-11-11T14:57:54Z)
• Where to Search: Measure the Prior-Structured Search Space of LLM Agents [0.8249180979158818]
  This paper proposes a compact formal theory that describes and measures LLM-assisted iterative search guided by domain priors.<n>We represent an agent as a fuzzy relation operator on inputs and outputs to capture feasible transitions.<n>We provide the simplest testable inferences and validate them via two instantiation.
  arXiv  Detail & Related papers  (2025-10-16T16:18:37Z)
• QAgent: A modular Search Agent with Interactive Query Understanding [25.147900132089777]
  Large language models excel at natural language tasks but are limited by their static parametric knowledge.<n>We propose a unified agentic RAG framework that employs a search agent for adaptive retrieval.<n> Experiments show QAgent excels at QA and serves as a plug-and-play module for real-world deployment.
  arXiv  Detail & Related papers  (2025-10-09T16:08:05Z)
• Rationale-Augmented Retrieval with Constrained LLM Re-Ranking for Task Discovery [4.061135251278187]
  Head Start programs utilizing GoEngage face significant challenges when new or rotating staff attempt to locate appropriate Tasks on the platform homepage.<n>These difficulties arise from domain-specific jargon, system-specific nomenclature, and the inherent limitations of lexical search in handling typos and varied word ordering.<n>We propose a pragmatic hybrid semantic search system that combines lightweight typo-tolerant lexical retrieval, embedding-based vector similarity, and constrained large language model (LLM) re-ranking.
  arXiv  Detail & Related papers  (2025-10-01T01:28:59Z)
• Implicit Reasoning in Large Language Models: A Comprehensive Survey [67.53966514728383]
  Large Language Models (LLMs) have demonstrated strong generalization across a wide range of tasks.<n>Recent studies have shifted attention from explicit chain-of-thought prompting toward implicit reasoning.<n>This survey introduces a taxonomy centered on execution paradigms, shifting the focus from representational forms to computational strategies.
  arXiv  Detail & Related papers  (2025-09-02T14:16:02Z)
• Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers [74.17516978246152]
  Large language models (LLMs) have been widely integrated into information retrieval to advance traditional techniques.<n>We propose EXSEARCH, an agentic search framework, where the LLM learns to retrieve useful information as the reasoning unfolds.<n>Experiments on four knowledge-intensive benchmarks show that EXSEARCH substantially outperforms baselines.
  arXiv  Detail & Related papers  (2025-05-26T15:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.