EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge

• URL: http://arxiv.org/abs/2606.13120v1
• Date: Thu, 11 Jun 2026 09:48:32 GMT
• Title: EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge
• Abstract summary: EvoBrowseComp is an evolving benchmark of 400 English and 400 Chinese complex questions.<n>EvoBrowseComp can be regularly updated to prevent data contamination.<n>It establishes a scalable paradigm for auto-updatable, high-difficulty benchmarking.
• Score: 53.44952808455985
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Search Agents -- large language models augmented with search tools -- have intensified the need for future-proof evaluation benchmarks. Existing benchmarks such as BrowseComp rely on static knowledge, making them vulnerable to test-set contamination and parametric memorization. Consequently, models can achieve high scores through fact recall rather than genuine retrieval, obscuring true browsing competence via reasoning shortcuts. In this paper, we introduce EvoBrowseComp, an evolving benchmark of 400 English and 400 Chinese contamination-free complex questions synthesized via live-web traversal. To collect these questions, we design a three-agent collaborative framework: (1) a QA synthesis agent that retrieves fresh knowledge from the live web to synthesize QA pairs; (2) an information filtering agent that filters retrieved knowledge in terms of credibility and popularity to block parametric shortcuts; and (3) a high-level guidance agent that formalizes questions into reasoning graphs to reduce logical redundancy and shortcuts in synthesized QA pairs. Because the framework supports fully automated synthesis, EvoBrowseComp can be regularly updated to prevent data contamination and maintain temporal freshness. Extensive experiments confirm its great difficulty, requiring broad horizontal search. It establishes a scalable paradigm for auto-updatable, high-difficulty benchmarking that keeps pace with both evolving world knowledge and advancing agent capabilities.

Related papers

• DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning [58.09293442211248]
  Deep Research (DR) has emerged as a new agentic paradigm to tackle complex, open-ended research tasks.<n>This report presents DuMate-DeepResearch, a multi-agent DR framework built on the Qianfan Agent Foundry.
  arXiv  Detail & Related papers  (2026-06-05T14:10:48Z)
• Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Design [39.31356016375221]
  Marco DeepResearch is a deep research agent optimized with a verification-centric framework design at three levels.<n>We introduce verification mechanisms to graph-based and agent-based QA synthesis to control question difficulty.<n>We design a verification-driven synthesis trajectory method that injects explicit verification patterns into training trajectories.<n>We use Marco DeepResearch itself as a verifier at inference time and effectively improve performance on challenging questions.
  arXiv  Detail & Related papers  (2026-03-30T12:42:02Z)
• Evaluating the Search Agent in a Parallel World [28.24678964635285]
  We introduce Mind-ParaWorld, a framework for evaluating Search Agents in a Parallel World.<n>A ParaWorld Law Model constructs a set of indivisible Atomic Facts and a unique ground-truth for each question.<n>During evaluation, instead of retrieving real-world results, the agent interacts with a ParaWorld Model.<n>We release MPW-Bench, an interactive benchmark spanning 19 domains with 1,608 instances.
  arXiv  Detail & Related papers  (2026-03-05T02:56:42Z)
• AgentIR: Reasoning-Aware Retrieval for Deep Research Agents [76.29382561831105]
  Deep Research agents generate explicit natural language reasoning before each search call.<n> Reasoning-Aware Retrieval embeds the agent's reasoning trace alongside its query.<n>DR- Synth generates Deep Research retriever training data from standard QA datasets.<n>AgentIR-4B achieves 68% accuracy with the open-weight agent Tongyi-DeepResearch.
  arXiv  Detail & Related papers  (2026-03-04T18:47:26Z)
• SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback [68.60326181052658]
  We propose an agentic pipeline that automatically generates high quality, difficulty-controlled deep search question-answer pairs.<n>Our pipeline, SAGE, consists of a data generator which proposes QA pairs and a search agent which attempts to solve the generated question.<n>Our intrinsic evaluation shows SAGE generates questions that require diverse reasoning strategies, while significantly increases the correctness and difficulty of the generated data.
  arXiv  Detail & Related papers  (2026-01-26T06:37:56Z)
• Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search [56.78490647843876]
  Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use.<n>We propose bfM-ASK, a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context.
  arXiv  Detail & Related papers  (2026-01-08T08:13:27Z)
• LiveSearchBench: An Automatically Constructed Benchmark for Retrieval and Reasoning over Dynamic Knowledge [31.40589987269264]
  We present LiveSearchBench, an automated pipeline for constructing retrieval-dependent benchmarks from recent knowledge updates.<n>Our method computes deltas between successive Wikidata snapshots, filters candidate triples for quality, and synthesizes natural-language questions at three levels of reasoning difficulty.<n> Experiments show a pronounced performance drop when models confront facts that post-date pretraining, with the gap most salient on multi-hop queries.
  arXiv  Detail & Related papers  (2025-11-03T10:00:49Z)
• WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents [72.28593628378991]
  WebResearcher is an iterative deep-research paradigm that reformulates deep research as a Markov Decision Process.<n>WebResearcher achieves state-of-the-art performance, even surpassing frontier proprietary systems.
  arXiv  Detail & Related papers  (2025-09-16T17:57:17Z)
• Open Data Synthesis For Deep Research [17.22470203913576]
  We formalize Deep Research tasks with verifiable answers as Hierarchical Constraint Satisfaction Problems.<n>Existing benchmarks (e.g., Natural Questions, HotpotQA) fail to capture this complexity.<n>We introduce InfoSeek, a scalable scalable complex Deep Research tasks.
  arXiv  Detail & Related papers  (2025-08-30T06:02:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.