Retrieval-Guided Fine-Tuning as Noisy Estimation: Risk bounds and Architectural Analysis

• URL: http://arxiv.org/abs/2609.14485v1
• Date: Sun, 13 Sep 2026 12:51:02 GMT
• Title: Retrieval-Guided Fine-Tuning as Noisy Estimation: Risk bounds and Architectural Analysis
• Abstract summary: Retrieval-Guided Fine-Tuning (RAG-FT) incorporates retrieved data directly into the training objective, but the statistical consequences of noisy retrieval during training remain theoretically undercharacterized.<n>We show that retrieval failure decays exponentially with task separation relative to noise, and derive explicit finite-sample conditions under which RAG-FT achieves lower risk than both target-only and full-corpus training.<n>We prove that the attention estimator's bias diverges as $(n2q)$ even under exact retrieval, while OLS risk remains $(d/n)$ for every noise
• Score: 1.381684956393831
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-Guided Fine-Tuning (RAG-FT) incorporates retrieved data directly into the training objective, but the statistical consequences of noisy retrieval during training remain theoretically undercharacterized. We study this question by modeling RAG-FT as an estimation problem in a multi-task linear regression framework, using an OLS proxy for single-layer linear self-attention to obtain finite-sample risk bounds. Under homoscedastic retrieval noise, we show that retrieval failure decays exponentially with task separation relative to noise, and derive explicit finite-sample conditions under which RAG-FT achieves lower risk than both target-only and full-corpus training. We then introduce a Distance-Proportional Noise (DPN) model, in which retrieval quality degrades with rank, and compare two estimators under the same retrieval process: the OLS proxy and the literal, uniform-weight forward pass of linear self-attention. We prove that the attention estimator's bias diverges as $Θ(n^{2q})$ even under exact retrieval, while OLS risk remains $Θ(d/n)$ for every noise exponent $q>0$. These results locate the instability not in noisy retrieval itself, but in the fixed, unweighted aggregation of the literal LSA forward pass, which reweighting by reliability empirically removes. We validate the predicted rate separation through direct simulation of the DPN model.

Related papers

• Precise: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models [56.67321805551389]
  Reinforcement learning (RL) has become an effective way to improve prompt alignment and perceptual quality in diffusion and flow-matching generators.<n>The sampler, controlling the exploration behavior and denoising dynamics, is thus part of the policy.<n>We propose a new sampler that balances effective exploration with stability.
  arXiv  Detail & Related papers  (2026-05-22T11:37:22Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Interleaved Resampling and Refitting: Data and Compute-Efficient Evaluation of Black-Box Predictors [15.386375612838371]
  We study the problem of evaluating the excess risk of large-scale empirical risk minimization under the square loss.<n>We assume only black-box access to the training algorithm and develop an efficient procedure for estimating the excess risk.
  arXiv  Detail & Related papers  (2026-03-15T04:41:20Z)
• Outlier-robust Autocovariance Least Square Estimation via Iteratively Reweighted Least Square [10.915853954229682]
  This paper proposes a novel outlier-robust ALS algorithm, termed ALS-IRLS, based on the iteratively reweighted least squares (IRLS) framework.<n>It significantly enhances downstream state estimation accuracy, outperforming existing outlier-robust Kalman filters.
  arXiv  Detail & Related papers  (2026-03-09T09:40:11Z)
• Sharp Convergence Rates for Masked Diffusion Models [53.117058231393834]
  We develop a total-variation based analysis for the Euler method that overcomes limitations.<n>Our results relax assumptions on score estimation, improve parameter dependencies, and establish convergence guarantees.<n>Overall, our analysis introduces a direct TV-based error decomposition along the CTMC trajectory and a decoupling-based path-wise analysis for FHS.
  arXiv  Detail & Related papers  (2026-02-26T00:47:51Z)
• Why Self-Training Helps and Hurts: Denoising vs. Signal Forgetting [6.369253528507392]
  Iterative self-training repeatedly refits a model on pseudo-labels generated by its own predictions.<n>We derive deterministic-equivalent recursions for the prediction risk and effective noise across iterations.
  arXiv  Detail & Related papers  (2026-02-15T07:28:12Z)
• MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics [72.00014675808228]
  Instability in Large Language Models evaluation process obscures true learning dynamics.<n>We introduce textbfMaP, a framework that integrates underlineMerging underlineand the underlinePass@k metric.<n>Experiments show that MaP yields significantly smoother performance curves, reduces inter-run variance, and ensures more consistent rankings.
  arXiv  Detail & Related papers  (2025-10-10T11:40:27Z)
• Outlier-Robust Linear System Identification Under Heavy-tailed Noise [2.07180164747172]
  We consider the problem of estimating the state transition matrix of a linear time-invariant (LTI) system.<n>We develop a novel robust system identification algorithm that relies on constructing multiple weakly-concentrated estimators.<n>We show that our algorithm and analysis technique can be easily extended to account for scenarios where an adversary can arbitrarily corrupt a small fraction of the collected trajectory data.
  arXiv  Detail & Related papers  (2024-12-31T12:53:02Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Exploring the Training Robustness of Distributional Reinforcement Learning against Noisy State Observations [7.776010676090131]
  State observations that an agent observes may contain measurement errors or adversarial noises, misleading the agent to take suboptimal actions or even collapse while training. In this paper, we study the training robustness of distributional Reinforcement Learning (RL), a class of state-of-the-art methods that estimate the whole distribution, as opposed to only the expectation, of the total return.
  arXiv  Detail & Related papers  (2021-09-17T22:37:39Z)
• Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learning [57.88785630755165]
  Empirical risk minimization (ERM) is the workhorse of machine learning, but its model-agnostic guarantees can fail when we use adaptively collected data. We study a generic importance sampling weighted ERM algorithm for using adaptively collected data to minimize the average of a loss function over a hypothesis class. For policy learning, we provide rate-optimal regret guarantees that close an open gap in the existing literature whenever exploration decays to zero.
  arXiv  Detail & Related papers  (2021-06-03T09:50:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.