What Survives Into Context: A Diagnostic for Budget-Constrained Multi-Hop RAG and When Submodular Evidence Packing Improves It

• URL: http://arxiv.org/abs/2607.00725v1
• Date: Wed, 01 Jul 2026 10:12:15 GMT
• Title: What Survives Into Context: A Diagnostic for Budget-Constrained Multi-Hop RAG and When Submodular Evidence Packing Improves It
• Abstract summary: We argue that document recall is the wrong quantity to optimize in a fixed reader-context budget.<n>We introduce answer-in-context, a diagnostic that measures whether a gold answer survives as a contiguous span in the packed reader context.<n>We map the scope of this win honestly: it requires the conjunction of (i) multi-hop complementary structure, (ii) retrieval that surfaces the evidence, (iii) a binding but not extreme budget, and (iv) a reader weak enough that evidence density, not reading capacity, is the bottleneck.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-augmented generation (RAG) under a fixed reader-context budget forces a selection problem: of the evidence retrieved, only a fraction can be shown to the reader. We argue that document recall -- the standard retrieval metric -- is the wrong quantity to optimize in this regime, and we make two contributions. First, as a general contribution, we introduce answer-in-context, a diagnostic that measures whether a gold answer survives as a contiguous span in the packed reader context (not the retrieved set). It predicts answer F1 better than recall (r=0.39-0.55 vs. about 0.31), separates answer quality roughly five-fold (0.60 vs. 0.12 on HotpotQA), and carries information beyond retrieval: it adds Delta R squared=0.17 over recall and shows a 4.6x EM gap even among questions where all gold was retrieved. We also confirm it interventionally: on 2WikiMultiHopQA a packing change that raises coverage but not answer-in-context yields no accuracy gain. Second, as a conditional contribution, we cast reader-context construction as budgeted monotone submodular maximization and build a packer that jointly optimizes relevance, query coverage, representativeness, and diversity. On HotpotQA with a 160-token budget and a 3B reader it beats a strong focused heuristic, MMR, and naive packing -- by up to +5.1 F1 at equal-or-lower token cost, across three seeds. Crucially, we map the scope of this win honestly: it requires the conjunction of (i) multi-hop complementary structure, (ii) retrieval that surfaces the evidence, (iii) a binding but not extreme budget, and (iv) a reader weak enough that evidence density, not reading capacity, is the bottleneck. A quantization-controlled reader-scale ladder (3B to 7B to 14B) shows the edge over the heuristic is absorbed by 7B and significantly reverses by 14B, while the diagnostic explains every boundary with a single variable.

Related papers

• PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents [85.82624962221026]
  We introduce PARSER, which decouples reading from reasoning.<n>A bank of lightweight subagents each bound to a single chunk read the entire document in parallel.<n>On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline.
  arXiv  Detail & Related papers  (2026-09-06T16:19:01Z)
• Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression [18.435649248662944]
  We show that when retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling.<n>We train a compact to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference.
  arXiv  Detail & Related papers  (2026-08-05T07:58:53Z)
• Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation [10.60293484295824]
  We formulate adaptive RAG as calibrated retrieval-budget allocation.<n>We calibrate sequence log-probability and prefix-logit uncertainty signals into probabilities of correctness.<n>For sequence log-probability, ECE drops from 0.275 to 0.062 on TriviaQA, 0.643 to 0.009 on NQ, and 0.711 to 0.031 on MS MARCO.
  arXiv  Detail & Related papers  (2026-06-29T08:36:33Z)
• Constrained Dominant Sets for Multimodal Document Question Answering [6.071121358322323]
  Long multimodal document question answering is limited by which evidence reaches the reader, rather than by the quantity retrieved.<n>This work introduces a retriever that selects evidence as a Constrained Dominant Set on a query-augmented affinity graph.
  arXiv  Detail & Related papers  (2026-06-05T13:24:46Z)
• Answer Presence Drives RAG Rewriting Gains [21.554301305644156]
  Retrieval-augmented QA pipelines often route retrieved passages through an LLM emphrewriter before a smaller reader.<n>We ask whether that lift is causally driven by the gold answer string appearing in the rewritten context.
  arXiv  Detail & Related papers  (2026-06-04T03:00:42Z)
• GridProbe: Posterior-Probing for Adaptive Test-Time Compute in Long-Video VLMs [3.9266376632068485]
  GridProbe is an efficient training-free posterior-probing inference paradigm.<n>It scores evidence in answer space using a frozen VLM's own reasoning.<n>It selects question-relevant frames adaptively, resulting in sub-quadratic attention cost with little to no accuracy loss.
  arXiv  Detail & Related papers  (2026-05-11T15:57:46Z)
• S3Mem: Structured Spatiotemporal Scene-Event Memory for Long-Horizon Interactive Question Answering [58.90783999951707]
  Long-horizon memory question answering often requires sparse evidence from heterogeneous histories.<n>We propose Structured Spatiotemporal Scene--Event Memory (S3Mem), a query-time memory interface that writes textual, visual, and agent-use histories into structured scene--event units and routes compact evidence packs to the reader.<n>Across LoCoMo, EMemBench Visual Games, and AMA-Bench, S3Mem provides a strong score--token trade-off, with the clearest gains on localized event, state, temporal, causal, or evidence provenance.
  arXiv  Detail & Related papers  (2026-04-10T07:49:10Z)
• Test-Time Strategies for More Efficient and Accurate Agentic RAG [58.44913384057518]
  Retrieval-Augmented Generation (RAG) systems face challenges with complex, multihop questions.<n>Such approaches can introduce inefficiencies, including repetitive retrieval of previously processed information.<n>We investigate test-time modifications to the Search-R1 pipeline to mitigate these identified shortcomings.
  arXiv  Detail & Related papers  (2026-03-12T19:18:59Z)
• SE-Search: Self-Evolving Search Agent via Memory and Dense Reward [87.79131676521656]
  Retrieval augmented generation (RAG) reduces hallucinations and factual errors in large language models (LLMs) by conditioning generation on retrieved external knowledge.<n>Existing methods often accumulate irrelevant or noisy documents and rely on sparse reinforcement learning signals.<n>We propose textbfSelf-textbfEvolving textbfSearch, a Self-Evolving Search agent that improves online search behavior through three components.
  arXiv  Detail & Related papers  (2026-02-06T09:14:07Z)
• $G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA [53.491241153213565]
  $G2$-Reader is a dual-graph system for multimodal question answering.<n>$G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)<n>On VisDoMBench across five multimodal domains, $G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)
  arXiv  Detail & Related papers  (2026-01-29T17:52:54Z)
• PluriHop: Exhaustive, Recall-Sensitive QA over Distractor-Rich Corpora [0.0]
  We introduce PluriHopWIND, a diagnostic multilingual dataset of 48 pluri-hop questions built from 191 real-world wind industry reports in German and English.<n>We show that PluriHopWIND is 8-40% more repetitive than other common datasets and thus has higher density of distractor documents.<n>We propose PluriHopRAG, a RAG architecture that follows a "check all documents individually, filter cheaply" approach.
  arXiv  Detail & Related papers  (2025-10-16T07:22:58Z)
• Faithful Chain-of-Thought Reasoning [51.21714389639417]
  Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of reasoning tasks. We propose Faithful CoT, a reasoning framework involving two stages: Translation and Problem Solving. This guarantees that the reasoning chain provides a faithful explanation of the final answer.
  arXiv  Detail & Related papers  (2023-01-31T03:04:26Z)
• C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References [47.20656650130518]
  Key challenge is how to construct a large amount of high-quality question-answer-context triplets without task-specific annotations. In this work, we automatically construct a large-scale corpus that meets all three criteria by consulting millions of references cited within Wikipedia. Our pretrained retriever leads to 2%-10% absolute gains in top-20 accuracy. And with our pretrained reader, the entire system improves by up to 4% in exact match.
  arXiv  Detail & Related papers  (2022-03-16T20:30:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.