Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

• URL: http://arxiv.org/abs/2608.06305v2
• Date: Fri, 07 Aug 2026 08:21:58 GMT
• Title: Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
• Abstract summary: Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query.<n>We argue that for an important class of documents -- financial statements, audit reports, regulatory returns -- this design is structurally unsound.<n>We propose READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query. We argue that for an important class of documents -- financial statements, audit reports, regulatory returns -- this design is structurally unsound, and we make the argument measurable. On a 780-page government financial report, 86.8% of content lines are table rows, thousands of near-identical figures compete in one embedding space, and a figure inherits its unit from a header a median of 13 lines above it -- so a chunk boundary routinely separates a number from whether it is in lakh or crore, an error of two orders of magnitude. A table-aware chunker built as a steelman fixes the unit problem but leaves 27-30% of numeric chunks with no fiscal-year header at every chunk size we tried. We propose READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations -- normalized lexical search, structural navigation, and bounded span reads -- exposed over the Model Context Protocol, so a trajectory is a replayable audit trail, not an opaque similarity score. On 51 verified questions READ answers 58.8% against dense retrieval's 15.7% (p_Holm = 2 x 10^-5) -- or 35.3% tuned, which READ still leads by 23.5 points (p_Holm = 0.017). An agent given the same loop but a top-k tool reaches only 27.5%, locating the gain in the interface rather than in iteration. We also report what the evidence does not support: BM25 is statistically indistinguishable from READ, so our result separates embedding-based from embedding-free retrieval, not agentic from lexical search.

Related papers

• Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems [67.04448659688579]
  We develop a query-independent 16-entry codebook that ranks the top decoded candidates in a single batched forward pass.<n>With no iterative search and no message passing at test time, Codebook Agent is the most accurate method on all six benchmarks.
  arXiv  Detail & Related papers  (2026-09-02T08:10:22Z)
• From Traceability to Justifiability: Accountability Structures in Agentic Software Engineering [0.0]
  We measure, from public material only, whether AI records can express a claim and whether they hold where declared.<n>Across 188 double-graded cells we found no platform whose default record emits a content identity of a behavioral system.<n>An instrument computes realized assurance depth from a pipeline's published exhaust alone and compares it with the declared depth.
  arXiv  Detail & Related papers  (2026-08-21T16:45:00Z)
• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• Structure-Aware Semantic Chunking with Title-Chain Prefixes: A 1600-Query Evaluation and the Measurement Trap in Text-Transform Ablations [4.342406076796542]
  We present a three-stage, chunk-side-only semantic chunking pipeline.<n>The title chain reuses the document's own header hierarchy instead of a generated summary.<n>The pipeline improves MRR@5 from 0.374 to 0.463 on the full set and from 0.828 to 0.925 on the answerable subset.
  arXiv  Detail & Related papers  (2026-08-01T19:01:46Z)
• Bridge Evidence: Static Retrieval Utility Does Not Predict Causal Utility in Multi-Step Agentic Search [3.976291254896486]
  Retrieval systems are trained and evaluated on a static idea of usefulness.<n>It breaks when a language model works as a search agent, issuing several queries and reasoning across turns.<n>We replay 1000 development questions and, for every document the agent read, delete it and re-run the rest of the trajectory from that point.
  arXiv  Detail & Related papers  (2026-07-16T17:48:29Z)
• GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents [0.12430801435092285]
  GDP_pdf is a benchmark built to measure document AI capabilities directly.<n>It consists of question-document pairs authored by working professionals in ten fields.<n>We report results for seventeen frontier models on the 100-item benchmark.
  arXiv  Detail & Related papers  (2026-07-13T07:44:07Z)
• LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents [2.3651168422805027]
  We frame this task as planning: an agent observes evidence collected so far, picks next document fragment to inspect, and stops when the evidence is sufficient.<n>The agent operates over a vectorless document tree using browse, read, and search tools, and maintains a dynamic knowledge graph (KG) as state.<n>On a 200-question benchmark over NuScale Final Safety Analysis Report (FSAR) documents, the system reaches 81.5% accuracy with a fidelity of 0.93.
  arXiv  Detail & Related papers  (2026-06-28T13:45:59Z)
• AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning [52.23991730630292]
  Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge.<n>We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer.<n>We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window.
  arXiv  Detail & Related papers  (2026-06-23T12:57:18Z)
• Novelty-Aware Agentic Retrieval: Comparing Research Contributions Through Structured Multi-Step Reasoning [0.0]
  Novelty-Aware Research Agent is a prototype agentic retrieval system.<n>It layers structured multi-step reasoning on a RAG pipeline through six typed-contract components.<n>It produces structured comparison artifacts: per-paper contribution records, paper-level overlaps, and a problem x method gap matrix.
  arXiv  Detail & Related papers  (2026-06-20T17:04:02Z)
• Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation [59.438696079345426]
  DICE is a training-free document-side strategy that splits documents into chunks, encodes them independently with a frozen model, and aggregates them back into a single vector.<n>On LongEmbed, DICE improves retrieval across four backbones, with the largest gains on slices beyond 4k tokens.<n>Across 12,779 filtered samples, DICE yields lower EDI than the single-vector baseline in 92.8% of cases.
  arXiv  Detail & Related papers  (2026-06-17T07:44:04Z)
• Towards Retrieving Interaction Spaces for Agentic Search [62.666902610940525]
  Recent direct corpus (DCI) work shows that agents can interact with the raw corpus through shell tools such as 5.4 interaction and file reads.<n>We argue that the role of retrieval for agentic search is not just to select documents that fit in the LLM context window, but to construct an interaction space.
  arXiv  Detail & Related papers  (2026-06-05T03:47:40Z)
• $G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA [53.491241153213565]
  $G2$-Reader is a dual-graph system for multimodal question answering.<n>$G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)<n>On VisDoMBench across five multimodal domains, $G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)
  arXiv  Detail & Related papers  (2026-01-29T17:52:54Z)
• ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge [50.93758649363798]
  Impliret is a benchmark that shifts the reasoning challenge to document-side processing.<n>We evaluate a range of sparse and dense retrievers, all of which struggle in this setting.
  arXiv  Detail & Related papers  (2025-06-17T11:08:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.