Equal Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents

• URL: http://arxiv.org/abs/2607.10198v1
• Date: Sat, 11 Jul 2026 08:22:35 GMT
• Title: Equal Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents
• Abstract summary: We argue that a commercial search API is better understood as a decision surface.<n>We test this claim with one frozen GPT-5.4 agent, two tools (search_web and fetch_page), and 100 questions from SEALQA-HARD.<n>A Kimi-K2.6 oracle labels every content element visible to the agent (URL, title, snippet, and fetched page, when fetched), producing 6,869 valid per-URL judgments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Search APIs are the fundamental retrieval layer for many agents and are often their most frequently used tool. Traditional search APIs provide URLs, titles, and snippets that preview website contents. Because full-page retrieval is token-intensive, agent retrieval architectures increasingly use progressive disclosure: the agent first sees snippets and then chooses whether to fetch full pages. In such systems, search API performance is often evaluated primarily by answer accuracy. We argue that a commercial search API is better understood as a decision surface: the ranked snippets, URLs, and metadata that determine whether an agent answers immediately, searches again, or spends tokens opening pages. We test this claim with one frozen GPT-5.4 agent, two tools (search_web and fetch_page), and 100 questions from SEALQA-HARD, varying only the search provider (Brave, Tavily, Firecrawl). A Kimi-K2.6 oracle labels every content element visible to the agent (URL, title, snippet, and fetched page, when fetched), producing 6,869 valid per-URL judgments. We use an audited correct-answer label, semantic match, which preserves exact matches while accepting harmless formatting and naming variants. Under this measure, the providers remain close (25, 25, 26 / 100), but their evidence economies differ sharply: Brave offers gold-answer-rich snippets, Tavily concentrates gold-supporting URLs at rank 1, and Firecrawl is associated with broader exploration under this fixed agent policy. We also introduce a surface contradiction-to-gold URL ratio, which varies from 0.92 to 2.59. Provider choice is therefore a retrieval-budget and policy decision, not merely a recall decision.

Related papers

• Pointing the Way, Hiding the Destination: Practical Private Dense Retrieval at Scale [19.48684574323907]
  Hosted retrieval-augmented generation (RAG) and semantic search allow users to query valuable provider-held corpora.<n>Existing cryptographic approaches either make this costly by processing the entire corpus for every query, or sacrifice quality for efficiency by scanning a few clusters.<n>This shortlist short-circuits full-corpus cryptographic search without sacrificing retrieval quality.
  arXiv  Detail & Related papers  (2026-08-26T12:46:36Z)
• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• Breadcrumbing Search Agents [118.20284823674382]
  LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk.<n>We show that the channel delivering search and page observations is a fragile security boundary.<n>Under a constrained tool-intermediary threat model, appending only one controlled result per query can substantially increase attack success.
  arXiv  Detail & Related papers  (2026-08-05T07:57:27Z)
• Search, Inspect, Fetch: Exploiting Structure-Aware Boolean Retrieval for Deep-Research Agents [50.35041517085015]
  Existing deep-research agents use a Search--Visit workflow that retrieves whole webpages without considering the structure they expose through titles, headings, sections, and metadata.<n>We introduce textscSieve, a search--inspect-fetch strategy driven by a Boolean Query Language (BQL)<n>It searches webpage fields to filter candidates, uses an interchangeable ranker to order them, presents structure-rich result cards for inspection, and fetches only selected sections.
  arXiv  Detail & Related papers  (2026-08-03T18:01:05Z)
• Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents [60.97609995940902]
  We propose textbfFetch-then-Explore for search agents.<n>It separates page selection from evidence extraction and keeps what it selects.<n>It leads BrowseComp accuracy at every backbone and generally matches or exceeds the baselines on WideSearch.
  arXiv  Detail & Related papers  (2026-08-03T11:58:21Z)
• Deep Agentic Search for Repository-Level Code Question Answering: An Empirical Study [0.18737506366172094]
  We compare the two approaches on SWE-QA, a benchmark for repository-level code question answering.<n>Semantic search answered 65.2% of questions correctly against 46.2% for deep agentic search.<n>Deep agentic search addresses a real problem and is now the preferred design in many code agents.
  arXiv  Detail & Related papers  (2026-08-02T21:36:59Z)
• FinSAgent: Corpus-Aligned Multi-Agent RAG Framework for Evidence-Grounded SEC Filing Question Answering [16.415865984091667]
  FinSAgent is an evidence-grounded multi-agent framework that reframes SEC filing QA as corpus-aligned retrieval planning.<n>FinSAgent improves retrieval coverage and answer correctness over strong single-agent and multi-agent baselines.
  arXiv  Detail & Related papers  (2026-07-20T16:03:15Z)
• Do Agents Need Semantic Metadata? A Comparative Study in Agentic Data Retrieval [3.6202634353482352]
  The rise of Large Language Models (LLMs) capable of navigating the unstructured web raises a fundamental question: Is semantic metadata still necessary for agentic data discovery?<n>We present a comparative analysis of agentic data retrieval across two distinct environments: a Baseline Agent searching billions of open-web documents, and a Semantic Agent leveraging a corpus of 90 million datasets using schema.org.<n>The Semantic Agent excels at retrieving actionable data, achieving a 44.9% higher precision for metadata-rich registries and a 46.6% higher precision for pages with machine-readable downloads among its returned results.
  arXiv  Detail & Related papers  (2026-05-27T17:46:43Z)
• LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know? [32.434901767447165]
  LLM-based search agents rely on intrinsic knowledge rather than external evidence.<n>LiveBrowseComp is a benchmark designed to evaluate agents beyond intrinsic coverage.<n>All evaluated agents fall below 2% closed-book accuracy, search-augmented scores drop by 25-40 points relative to BrowseComp, and prior model rankings no longer reliably predict performance.
  arXiv  Detail & Related papers  (2026-05-27T16:39:57Z)
• Revisiting Text Ranking in Deep Research [24.324221566628125]
  Black-box web search APIs hinder systematic analysis of search components.<n>We reproduce a selection of key findings and best practices for IR text ranking methods in the deep research setting.
  arXiv  Detail & Related papers  (2026-02-25T00:18:07Z)
• To Search or Not to Search: Aligning the Decision Boundary of Deep Search Agents via Causal Intervention [61.82680155643223]
  We identify the root cause of misaligned decision boundaries, the threshold determining when accumulated information suffices to answer.<n>This causes over-search (redundant searching despite sufficient knowledge) and under-search (premature termination yielding incorrect answers.<n>We propose a comprehensive framework comprising two key components. First, we introduce causal intervention-based diagnosis that identifies boundary errors.<n>Second, we develop Decision Boundary Alignment for Deep Search agents (DAS)<n>Our DAS method effectively calibrates these boundaries, mitigating both over-search and under-search to achieve substantial gains in accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-02-03T09:29:06Z)
• Beyond Browsing: API-Based Web Agents [58.39129004543844]
  API-Based Agents outperform web Browsing Agents in experiments on WebArena.<n>Hybrid Agents out-perform both others nearly uniformly across tasks.<n>Results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.
  arXiv  Detail & Related papers  (2024-10-21T19:46:06Z)
• Database-Augmented Query Representation for Information Retrieval [71.41745087624528]
  We present a novel retrieval framework called Database-Augmented Query representation (DAQu)<n>DAQu augments the original query with various (query-related) metadata across multiple tables.<n>We validate our DAQu in diverse retrieval scenarios, demonstrating that it significantly enhances overall retrieval performance.
  arXiv  Detail & Related papers  (2024-06-23T05:02:21Z)
• Large Language Models are Built-in Autoregressive Search Engines [19.928494069013485]
  Large language models (LLMs) can follow human instructions to directly generate URLs for document retrieval. LLMs can generate Web URLs where nearly 90% of the corresponding documents contain correct answers to open-domain questions.
  arXiv  Detail & Related papers  (2023-05-16T17:04:48Z)
• Enhanced vectors for top-k document retrieval in Question Answering [0.0]
  We propose a different approach that retrieves the evidence documents efficiently and accurately. We do so by assigning each document (or passage in our case), a unique identifier and using them to create dense vectors. This approach enables efficient creation of real-time query vectors in 4 milliseconds.
  arXiv  Detail & Related papers  (2022-10-08T07:44:24Z)
• Adaptive Information Seeking for Open-Domain Question Answering [61.39330982757494]
  We propose a novel adaptive information-seeking strategy for open-domain question answering, namely AISO. According to the learned policy, AISO could adaptively select a proper retrieval action to seek the missing evidence at each step. AISO outperforms all baseline methods with predefined strategies in terms of both retrieval and answer evaluations.
  arXiv  Detail & Related papers  (2021-09-14T15:08:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.