SAAS: Self-Aware Reinforcement Learning for Over-Search Mitigation in Agentic Search

• URL: http://arxiv.org/abs/2605.29796v2
• Date: Fri, 29 May 2026 02:17:31 GMT
• Title: SAAS: Self-Aware Reinforcement Learning for Over-Search Mitigation in Agentic Search
• Abstract summary: Agentic search enables LLMs to solve complex multi-hop questions through iterative reasoning and external search.<n>Despite the effectiveness, these systems often suffer from a critical limitation in practice.<n>We propose SAAS, a novel RL framework designed to cultivate dynamic self-awareness that precisely regulates search behavior without compromising accuracy.
• Score: 38.532946868233736
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic search enables LLMs to solve complex multi-hop questions through iterative reasoning and external search. Despite the effectiveness, these systems often suffer from a critical limitation in practice: agents fail to recognize their own knowledge boundaries, blindly triggering searches when internal knowledge suffices and failing to terminate search even when adequate evidence has been collected. The lack of self-awareness leads to severe \textbf{over-search}, incurring substantial inference latency and prohibitive computational cost. To this end, we propose SAAS, a novel RL framework designed to cultivate dynamic self-awareness that precisely regulates search behavior without compromising accuracy. SAAS introduces three key components: (i) a search boundary modeling mechanism, which identifies the search boundary under the evolving policy by contrasting search-disabled and search-enabled rollouts; (ii) a boundary-aware reward module, which translates this boundary awareness into trajectory-level penalties, suppressing unnecessary and redundant searches; and (iii) a stage-wise optimization strategy, which leverages a sequential curriculum to prioritize reasoning over search regularization, thereby avoiding reward hacking. Extensive experiments demonstrate that SAAS substantially reduces over-search, while maintaining accuracy. Our code and implementation details are released at https://github.com/XMUDeepLIT/SAAS.

Related papers

• R$^2$-Searcher: Calibrating Retrieval and Reasoning Boundaries for Agentic Search [59.034878430183305]
  R$2$-Searcher explicitly explores and calibrates the retrieval and reasoning boundaries.<n>We show that R$2$-Searcher significantly outperforms state-of-the-art agentic search methods in answer accuracy and retrieval-reasoning quality.
  arXiv  Detail & Related papers  (2026-06-26T19:44:34Z)
• AutoSearch: Adaptive Search Depth for Efficient Agentic RAG via Reinforcement Learning [52.305422887002656]
  We propose a reinforcement learning framework that evaluates each search step via self-generated intermediate answers.<n>By a self-answering mechanism, AutoSearch identifies the minimal sufficient search depth and promotes efficient search.<n>Experiments show that AutoSearch achieves a superior accuracy-efficiency trade-off, alleviating over-searching while preserving search quality.
  arXiv  Detail & Related papers  (2026-04-19T09:05:48Z)
• SIGHT: Reinforcement Learning with Self-Evidence and Information-Gain Diverse Branching for Search Agent [39.43590030917357]
  SIGHT is a framework that enhances search-based reasoning through Self-Evidence Support and Information-Gain Driven Diverse Branching.<n>SIGHT distills search results into high-fidelity evidence via SES and calculates an Information Gain score to pinpoint pivotal states.<n> Experiments on single-hop and multi-hop QA benchmarks demonstrate that SIGHT significantly outperforms existing approaches.
  arXiv  Detail & Related papers  (2026-02-12T04:16:55Z)
• To Search or Not to Search: Aligning the Decision Boundary of Deep Search Agents via Causal Intervention [61.82680155643223]
  We identify the root cause of misaligned decision boundaries, the threshold determining when accumulated information suffices to answer.<n>This causes over-search (redundant searching despite sufficient knowledge) and under-search (premature termination yielding incorrect answers.<n>We propose a comprehensive framework comprising two key components. First, we introduce causal intervention-based diagnosis that identifies boundary errors.<n>Second, we develop Decision Boundary Alignment for Deep Search agents (DAS)<n>Our DAS method effectively calibrates these boundaries, mitigating both over-search and under-search to achieve substantial gains in accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-02-03T09:29:06Z)
• AdaSearch: Balancing Parametric Knowledge and Search in Large Language Models via Reinforcement Learning [61.974530499621274]
  Overreliance on search introduces unnecessary cost and risks exposure to noisy or malicious content.<n>We propose a two-stage, outcome-driven RL framework that disentangles problem solving from the decision of whether to invoke search.<n>AdaSearch substantially improves knowledge-boundary awareness, reduces unnecessary search calls, preserves strong task performance, and offers more transparent, interpretable decision behaviors.
  arXiv  Detail & Related papers  (2025-12-18T18:50:01Z)
• Beyond Outcome Reward: Decoupling Search and Answering Improves LLM Agents [19.31471304268234]
  We introduce DeSA (Decoupling Search-and-Answering), a simple two-stage training framework that explicitly separates search optimization from answer generation.<n>Across seven QA benchmarks, DeSA-trained agents consistently improve search behaviors, delivering substantially higher search recall and answer accuracy than outcome-only baselines.
  arXiv  Detail & Related papers  (2025-10-06T11:09:45Z)
• RE-Searcher: Robust Agentic Search with Goal-oriented Planning and Self-reflection [55.125987985864896]
  We present a systematic analysis that quantifies how environmental complexity induces fragile search behaviors.<n>We propose a simple yet effective approach to instantiate a search agent, RE-Searcher.<n>This combination of goal-oriented planning and self-reflection enables RE-Searcher to resist spurious cues in complex search environments.
  arXiv  Detail & Related papers  (2025-09-30T10:25:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.