Replace, Don't Expand: Mitigating Context Dilution in Multi-Hop RAG via Fixed-Budget Evidence Assembly

• URL: http://arxiv.org/abs/2512.10787v1
• Date: Thu, 11 Dec 2025 16:31:29 GMT
• Title: Replace, Don't Expand: Mitigating Context Dilution in Multi-Hop RAG via Fixed-Budget Evidence Assembly
• Abstract summary: RAG systems often fail on multi-hop queries when the initial retrieval misses a bridge fact.<n>We propose textbfSEAL-RAG, a training-free controller that adopts a textbfreplace, don't expand'' strategy to fight context dilution.<n>We evaluate SEAL-RAG against faithful re-implementations of Basic RAG, CRAG, Self-RAG, and Adaptive-$k$ in a shared environment.
• Score: 1.2891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-Augmented Generation (RAG) systems often fail on multi-hop queries when the initial retrieval misses a bridge fact. Prior corrective approaches, such as Self-RAG, CRAG, and Adaptive-$k$, typically address this by \textit{adding} more context or pruning existing lists. However, simply expanding the context window often leads to \textbf{context dilution}, where distractors crowd out relevant information. We propose \textbf{SEAL-RAG}, a training-free controller that adopts a \textbf{``replace, don't expand''} strategy to fight context dilution under a fixed retrieval depth $k$. SEAL executes a (\textbf{S}earch $\rightarrow$ \textbf{E}xtract $\rightarrow$ \textbf{A}ssess $\rightarrow$ \textbf{L}oop) cycle: it performs on-the-fly, entity-anchored extraction to build a live \textit{gap specification} (missing entities/relations), triggers targeted micro-queries, and uses \textit{entity-first ranking} to actively swap out distractors for gap-closing evidence. We evaluate SEAL-RAG against faithful re-implementations of Basic RAG, CRAG, Self-RAG, and Adaptive-$k$ in a shared environment on \textbf{HotpotQA} and \textbf{2WikiMultiHopQA}. On HotpotQA ($k=3$), SEAL improves answer correctness by \textbf{+3--13 pp} and evidence precision by \textbf{+12--18 pp} over Self-RAG. On 2WikiMultiHopQA ($k=5$), it outperforms Adaptive-$k$ by \textbf{+8.0 pp} in accuracy and maintains \textbf{96\%} evidence precision compared to 22\% for CRAG. These gains are statistically significant ($p<0.001$). By enforcing fixed-$k$ replacement, SEAL yields a predictable cost profile while ensuring the top-$k$ slots are optimized for precision rather than mere breadth. We release our code and data at https://github.com/mosherino/SEAL-RAG.

Related papers

• SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization [0.21485350418225238]
  Fine-tuned encoders deployed across heterogeneous NLP tasks face three compounding problems: mismatched inductive biases, class-imbalance corruption of feature statistics, and no mechanism to condition attention on external lexical knowledge.<n>We introduce textbfsurgellm, a unified transformer framework that addresses each with a dedicated lightweight module.
  arXiv  Detail & Related papers  (2026-06-23T07:47:21Z)
• Isolating LLM Alignment from Regex: Zero Coverage and Metric-Dependent Divergence Under Adversarial Mutation [51.56484100374058]
  Previous work found no measurable coverage gain from adding a live Gemini backend behind an active filter.<n>We introduce $L_5$-no-regex -- identical to $L_4$-real (Gemini-2.5-flash, token-budget cap, rate limit, output scrub) but with the nine-pattern filter disabled.<n>We evaluate it against adversarial probes across three sub-languages.
  arXiv  Detail & Related papers  (2026-06-12T18:54:24Z)
• RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding [80.12789199134511]
  Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency.<n>We propose $textbfRACER, a lightweight and training-free method that integrates retrieved exact patterns with logit-driven future cues.<n>Experiments on Spec-Bench, HumanEval, and MGSM-ZH demonstrate that RACER consistently accelerates inference, achieving more than $2times$ speedup over autoregressive decoding.
  arXiv  Detail & Related papers  (2026-04-16T11:23:55Z)
• PAR$^2$-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering [57.89576196160413]
  Large language models (LLMs) remain brittle on multi-hop question answering (MHQA)<n>We propose textbfPlanned Active Retrieval and Reasoning RAG (PAR$2$-RAG), a two-stage framework that separates emphcoverage from emphcommitment.
  arXiv  Detail & Related papers  (2026-03-30T23:52:54Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• One Good Source is All You Need: Near-Optimal Regret for Bandits under Heterogeneous Noise [49.12618706309658]
  Source-Optimistic Adaptive Regret minimization (SOAR) is a novel algorithm that prunes high-variance sources using sharp variance-concentration bounds.<n>We show it achieves the optimal instance-dependent regret of standard single-source MAB with variance $*2$.<n>Our theoretical bounds represent a significant improvement over some proposed baselines.
  arXiv  Detail & Related papers  (2026-02-16T05:25:06Z)
• FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Models [27.658955798426323]
  AdamW has become one of the most effective generalizations for training large-scale models.<n>We propose the first underlineFederated underlineAdamW algorithm, called textttFedAdamW, for training and fine-tuning various large models.
  arXiv  Detail & Related papers  (2025-10-31T14:04:43Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Directed Information $γ$-covering: An Information-Theoretic Framework for Context Engineering [2.541246776283285]
  We introduceDirected Information $gamma$-covering, a framework for redundancy-aware context engineering.<n>Building the $gamma$-cover is emphquery-agnostic: it incurs no online cost and can be computed once offline and amortized across all queries.<n>These results establish DI $gamma$-covering as a principled, self-organizing backbone for modern LLM pipelines.
  arXiv  Detail & Related papers  (2025-09-30T02:41:11Z)
• Distribution-Aware Feature Selection for SAEs [1.2396474483677118]
  TopK SAE reconstructs each token from its K most active latents.<n> BatchTopK addresses this limitation by selecting top activations across a batch of tokens.<n>This improves average reconstruction but risks an "activation lottery"
  arXiv  Detail & Related papers  (2025-08-29T04:42:17Z)
• FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA [68.44043212834204]
  Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in learning (FL)<n>Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in learning (FL)
  arXiv  Detail & Related papers  (2025-05-19T07:32:56Z)
• Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimization [77.3396841985172]
  We provide a unified analysis of two-timescale gradient ascent (TTGDA) for solving structured non minimax optimization problems.<n>Our contribution is to design TTGDA algorithms are effective beyond the setting.
  arXiv  Detail & Related papers  (2024-08-21T20:14:54Z)
• An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encoding [25.20222970947923]
  We propose a method to extend the context length of pre-trained large language models (LLMs) $textttCREAM$ interpolates positional encodings by manipulating position indices. Experiments show that $textttCREAM$ successfully extends LLMs to the target length for both Base and Chat versions of $texttLlama2-7B$ with "Never Miss A Beat"
  arXiv  Detail & Related papers  (2024-06-11T10:35:49Z)
• Vision Transformer with Sparse Scan Prior [24.78780746169092]
  We propose a textbfSparse textbfScan textbfSelf-textbfAttention mechanism ($rmS3rmA$)<n>This mechanism predefines a series of Anchors of Interest for each token and employs local attention to efficiently model the spatial information around these anchors.<n>Building on $rmS3rmA$, we introduce the textbfSparse textbfScan textbfVision
  arXiv  Detail & Related papers  (2024-05-22T04:34:36Z)
• Distribution-Independent Regression for Generalized Linear Models with Oblivious Corruptions [49.69852011882769]
  We show the first algorithms for the problem of regression for generalized linear models (GLMs) in the presence of additive oblivious noise. We present an algorithm that tackles newthis problem in its most general distribution-independent setting. This is the first newalgorithmic result for GLM regression newwith oblivious noise which can handle more than half the samples being arbitrarily corrupted.
  arXiv  Detail & Related papers  (2023-09-20T21:41:59Z)
• On the Interplay Between Misspecification and Sub-optimality Gap in Linear Contextual Bandits [76.2262680277608]
  We study linear contextual bandits in the misspecified setting, where the expected reward function can be approximated by a linear function class. We show that our algorithm enjoys the same gap-dependent regret bound $tilde O (d2/Delta)$ as in the well-specified setting up to logarithmic factors.
  arXiv  Detail & Related papers  (2023-03-16T15:24:29Z)
• A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits [77.09836892653176]
  We study federated contextual linear bandits, where $M$ agents cooperate with each other to solve a global contextual linear bandit problem with the help of a central server. We consider the asynchronous setting, where all agents work independently and the communication between one agent and the server will not trigger other agents' communication. We prove that the regret of textttFedLinUCB is bounded by $tildeO(dsqrtsum_m=1M T_m)$ and the communication complexity is $tildeO(dM
  arXiv  Detail & Related papers  (2022-07-07T06:16:19Z)
• Pick up the PACE: Fast and Simple Domain Adaptation via Ensemble Pseudo-Labeling [16.14184145802016]
  Domain Adaptation (DA) has received widespread attention from deep learning researchers in recent years because of its potential to improve test accuracy with out-of-distribution labeled data. In this work, we propose a fast and simple DA method consisting of three stages: (1) domain alignment by covariance matching, (2) pseudo-labeling, and (3) ensembling. PACE exceeds previous state-of-the-art by $textbf5 - 10 %$ on most benchmark adaptation tasks without training a neural network.
  arXiv  Detail & Related papers  (2022-05-26T17:20:08Z)
• Semi-supervised Active Regression [21.51757844385258]
  This paper studies the use of partially labelled, potentially biased data for learning tasks. The learner has access to a dataset $X in mathbbRd | X min_beta in mathbbRd | X beta - Y |2 end2 equation while making few additional label queries.
  arXiv  Detail & Related papers  (2021-06-12T03:28:43Z)
• Learning to increase matching efficiency in identifying additional b-jets in the $\ ext{t}\ar{\ ext{t}}\ ext{b}\ar{\ ext{b}}$ process [0.0]
  The $texttbartextbbartextb$ process produces a top quark pair in association with a b quark pair. We adopt deep learning methods to learn from data to identify the additional b-jets from the $texttbartextbbartextb$ events.
  arXiv  Detail & Related papers  (2021-03-16T15:16:29Z)
• Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap [84.66885506098724]
  This paper presents a new model-free algorithm for episodic finite-horizon Markov Decision Processes (MDP), Adaptive Multi-step Bootstrap (AMB) We show AMB achieves a gap-dependent regret bound that only scales with the sum of the inverse of the sub-optimality gaps. We also show AMB suffers an additional $frac|Z_mul|Delta_min$ regret, where $Z_mul$ is the set of state-action pairs $(s,a)$'s satisfying $a$ is a non-unique optimal action for
  arXiv  Detail & Related papers  (2021-02-09T07:46:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.