Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search

• URL: http://arxiv.org/abs/2607.15253v1
• Date: Thu, 16 Jul 2026 17:48:29 GMT
• Title: Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search
• Abstract summary: Retrieval systems are trained and evaluated on a static idea of usefulness.<n>It breaks when a language model works as a search agent, issuing several queries and reasoning across turns.<n>We replay 1000 development questions and, for every document the agent read, delete it and re-run the rest of the trajectory from that point.
• Score: 3.976291254896486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval systems are trained and evaluated on a static idea of usefulness: hand a document and a question to a reader model, see whether the answer improves, and score the document accordingly. The idea holds up when a document is read on its own. It breaks when a language model works as a search agent, issuing several queries and reasoning across turns, because a document can matter for what it lets the agent do next rather than for what it says about the current question. We measure that gap rather than argue it. Using a ReAct style agent over HotpotQA, we replay 1000 development questions and, for every document the agent read, delete it and re-run the rest of the trajectory from that point. Comparing the original run against its counterfactual gives a Counterfactual Trajectory Utility (CTU) score from three deltas: final answer quality, next query retrieval quality, and turn count. Crossing CTU against Static RAG Utility (SRU) over 23,322 document observations, the two are close to statistically independent (Spearman rho = -0.026). Roughly a third of the documents the agent reads are causally load bearing while looking useless to a static reader; we call these bridge documents. The pattern survives when the reader based axis is swapped for a BM25 and cross encoder proxy, giving a bridge cell of 27.2% on an evenly spread axis. A second experiment pins down the mechanism. Using the Observable Entity Relevance (OER) measure from prior work, entities that discriminate relevant from non-relevant candidates appear in the agent's next query 4.02 times more often than entities found only in non-relevant documents (6.1% vs 1.5%, n = 227,139). A bridge document earns its keep by handing the agent a discriminative entity that redirects the search. Static relevance and causal usefulness are different quantities in agentic retrieval, and optimizing the first does not deliver the second.

Related papers

• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations [0.0]
  Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query.<n>We argue that for an important class of documents -- financial statements, audit reports, regulatory returns -- this design is structurally unsound.<n>We propose READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations.
  arXiv  Detail & Related papers  (2026-08-06T17:23:13Z)
• Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents [60.97609995940902]
  We propose textbfFetch-then-Explore for search agents.<n>It separates page selection from evidence extraction and keeps what it selects.<n>It leads BrowseComp accuracy at every backbone and generally matches or exceeds the baselines on WideSearch.
  arXiv  Detail & Related papers  (2026-08-03T11:58:21Z)
• Is Progressive Disclosure All You Need for Long-Context Agents? [6.214989743502112]
  Agentic AI suggests a broader option, giving the agent the document path and letting it decide how and what to read.<n>Agent Skills, a standard for packaging expertise into an agent loads on demand, supply a ready mechanism: progressive disclosure.<n>We run the first controlled study of the pattern, comparing raw-document navigation and several designs of Agent Skills packs against a classical hybrid retriever.
  arXiv  Detail & Related papers  (2026-07-20T06:35:32Z)
• Progressive Disclosure for LLM-Maintained Wiki Knowledge Bases: a Preregistered Ablation [0.0]
  We run a preregistered ablation in which four versions of the corpus differ only in how the agent reaches the content.<n>We grade answers blind against verified gold references with a cross-family judge.<n>The study doubles as a case study in evaluation validity, applying threat-to-validity discipline to the tooling that produced it.
  arXiv  Detail & Related papers  (2026-07-06T01:03:27Z)
• Constrained Dominant Sets for Multimodal Document Question Answering [6.071121358322323]
  Long multimodal document question answering is limited by which evidence reaches the reader, rather than by the quantity retrieved.<n>This work introduces a retriever that selects evidence as a Constrained Dominant Set on a query-augmented affinity graph.
  arXiv  Detail & Related papers  (2026-06-05T13:24:46Z)
• Improving Document Retrieval Coherence for Semantically Equivalent Queries [63.97649988164166]
  We propose a variation of the Multi-Negative Ranking loss for training DR that improves the coherence of models in retrieving the same documents.<n>The loss penalizes discrepancies between the top-k ranked documents retrieved for diverse but semantic equivalent queries.
  arXiv  Detail & Related papers  (2025-08-11T13:34:59Z)
• Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence [56.09494651178128]
  Retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG)<n>We quantify the impact of biases, such as a preference for shorter documents, on retrievers like Dragon+ and Contriever.<n>We uncover major vulnerabilities, showing retrievers favor shorter documents, early positions, repeated entities, and literal matches, all while ignoring the answer's presence!
  arXiv  Detail & Related papers  (2025-03-06T23:23:13Z)
• DAPR: A Benchmark on Document-Aware Passage Retrieval [57.45793782107218]
  We propose and name this task emphDocument-Aware Passage Retrieval (DAPR) While analyzing the errors of the State-of-The-Art (SoTA) passage retrievers, we find the major errors (53.5%) are due to missing document context. Our created benchmark enables future research on developing and comparing retrieval systems for the new task.
  arXiv  Detail & Related papers  (2023-05-23T10:39:57Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.