LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?

• URL: http://arxiv.org/abs/2605.28721v1
• Date: Wed, 27 May 2026 16:39:57 GMT
• Title: LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know?
• Abstract summary: LLM-based search agents rely on intrinsic knowledge rather than external evidence.<n>LiveBrowseComp is a benchmark designed to evaluate agents beyond intrinsic coverage.<n>All evaluated agents fall below 2% closed-book accuracy, search-augmented scores drop by 25-40 points relative to BrowseComp, and prior model rankings no longer reliably predict performance.
• Score: 32.434901767447165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Are LLM-based search agents genuinely searching, or using the web to verify what they already know? We study this question on BrowseComp with three diagnostics. Our analysis reveals Intrinsic Knowledge Dependence (IKD): even with tool access, agents often rely on intrinsic knowledge -- information encoded in the model before retrieval -- rather than on external evidence. Agents answer up to 44.5% of BrowseComp questions without tools, generate more than half of their search queries from internally produced hypotheses rather than retrieved leads, and perform worse than closed-book baselines when answer-supporting evidence is removed. These results suggest that static search benchmarks can reward memory-backed verification rather than evidence-driven discovery, conflating what agents already know with what they can find. We then introduce LiveBrowseComp, a deep-search benchmark designed to evaluate agents beyond intrinsic coverage. It contains 335 human-authored questions whose answers depend on facts published within the 90 days preceding benchmark construction, drawn from six updated sources and filtered to exclude globally salient events. On LiveBrowseComp, all evaluated agents fall below 2% closed-book accuracy, search-augmented scores drop by 25-40 points relative to BrowseComp, and prior model rankings no longer reliably predict performance. LiveBrowseComp is available at https://huggingface.co/datasets/Forival/LiveBrowseComp.

Related papers

• Projecting BrowseComp-Plus onto ClimbMix: Toward More Realistic Corpora for Agentic Search [40.897106650424035]
  The BrowseComp-Plus benchmark disentangled the evaluation of agentic search by replacing opaque web search with a fixed corpus.<n>We introduce $textBrowseComp-Plus_textCM$, which keeps the BrowseComp-Plus questions but relocates their evidence to ClimbMix.<n>Our main contribution is the projection pipeline that makes this possible.
  arXiv  Detail & Related papers  (2026-08-20T17:54:48Z)
• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• Diagnosing Search Behavior and Failure Modes in Long-Horizon Search Agents [63.2133766267587]
  Deep search agents answer difficult information-seeking questions by iteratively issuing search queries to gather supporting evidence.<n>We study these questions through a trajectory-level diagnosis of long-horizon search agents.<n>We evaluate the evidence retrieved at each search step and separate two stages of agent behavior: what evidence an agent retrieves and how effectively it uses that evidence.
  arXiv  Detail & Related papers  (2026-08-03T08:46:48Z)
• EigentSearch-Q+: Enhancing Deep Research Agents with Structured Reasoning Tools [12.61248869302151]
  We introduce Q+, a set of query and evidence processing tools that make web search more deliberate.<n>We integrate Q+ into the browser sub-agent of Eigent, an open-source, production-ready multi-agent workforce.<n>Across four benchmarks, Q+ improves Eigent's browser agent benchmark-size-weighted average accuracy by 3.0, 3.8, and 0.6 percentage points (pp)
  arXiv  Detail & Related papers  (2026-04-09T07:47:31Z)
• iAgentBench: Benchmarking Sensemaking Capabilities of Information-Seeking Agents on High-Traffic Topics [9.25340189071758]
  We present iAgentBench, a dynamic ODQA benchmark for cross-source sensemaking.<n>iAgentBench draws seed topics from real-world attention signals and uses common user intent patterns to construct user-like questions.<n>Each instance is released with traceable evidence and auditable intermediate artifacts that support contamination checks.
  arXiv  Detail & Related papers  (2026-03-04T22:40:08Z)
• AgentIR: Reasoning-Aware Retrieval for Deep Research Agents [76.29382561831105]
  Deep Research agents generate explicit natural language reasoning before each search call.<n> Reasoning-Aware Retrieval embeds the agent's reasoning trace alongside its query.<n>DR- Synth generates Deep Research retriever training data from standard QA datasets.<n>AgentIR-4B achieves 68% accuracy with the open-weight agent Tongyi-DeepResearch.
  arXiv  Detail & Related papers  (2026-03-04T18:47:26Z)
• SE-Search: Self-Evolving Search Agent via Memory and Dense Reward [87.79131676521656]
  Retrieval augmented generation (RAG) reduces hallucinations and factual errors in large language models (LLMs) by conditioning generation on retrieved external knowledge.<n>Existing methods often accumulate irrelevant or noisy documents and rely on sparse reinforcement learning signals.<n>We propose textbfSelf-textbfEvolving textbfSearch, a Self-Evolving Search agent that improves online search behavior through three components.
  arXiv  Detail & Related papers  (2026-02-06T09:14:07Z)
• SafeSearch: Do Not Trade Safety for Utility in LLM Search Agents [14.471045017602428]
  Large language model (LLM) based search agents iteratively generate queries, retrieve external information, and answer open-domain questions.<n>While researchers have primarily focused on improving their utility, their safety behaviors remain underexplored.<n>We present SafeSearch, a multi-objective reinforcement learning approach that couples a final-output safety/utility reward with a novel query-level shaping term.
  arXiv  Detail & Related papers  (2025-10-19T21:47:19Z)
• InfoAgent: Advancing Autonomous Information-Seeking Agents [143.15973604285304]
  We introduce InfoAgent, a deep research agent powered by an innovative data synthesis pipeline and orchestrated web search tools.<n>With our methods, InfoAgent achieves 15.3% accuracy on BrowseComp, 29.2% on BrowseComp-ZH, and 40.4% on Xbench-DS.
  arXiv  Detail & Related papers  (2025-09-29T17:59:57Z)
• BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent [74.10138164281618]
  BrowseComp-Plus is a benchmark derived from BrowseComp, employing a fixed, carefully curated corpus.<n>This benchmark allows comprehensive evaluation and disentangled analysis of deep research agents and retrieval methods.
  arXiv  Detail & Related papers  (2025-08-08T17:55:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.