When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering

• URL: http://arxiv.org/abs/2601.19827v2
• Date: Thu, 05 Feb 2026 15:59:52 GMT
• Title: When Iterative RAG Beats Ideal Evidence: A Diagnostic Study in Scientific Multi-hop Question Answering
• Abstract summary: We study whether synchronized iterative retrieval and reasoning can surpass an idealized static upper bound (Gold Context) RAG.<n>We benchmark eleven state-of-the-art LLMs under three regimes: (i) No Context, measuring reliance on parametric memory; (ii) Gold Context, where all evidence is supplied at once; and (iii) Iterative RAG, a training-free oracle controller that alternates retrieval, hypothesis refinement, and evidence-aware stopping.<n>Across models, Iterative RAG consistently outperforms Gold Context, with gains up to 25.6 percentage points, especially for non-reasoning
• Score: 0.2796197251957245
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-Augmented Generation (RAG) extends large language models (LLMs) beyond parametric knowledge, yet it is unclear when iterative retrieval-reasoning loops meaningfully outperform static RAG, particularly in scientific domains with multi-hop reasoning, sparse domain knowledge, and heterogeneous evidence. We provide the first controlled, mechanism-level diagnostic study of whether synchronized iterative retrieval and reasoning can surpass an idealized static upper bound (Gold Context) RAG. We benchmark eleven state-of-the-art LLMs under three regimes: (i) No Context, measuring reliance on parametric memory; (ii) Gold Context, where all oracle evidence is supplied at once; and (iii) Iterative RAG, a training-free controller that alternates retrieval, hypothesis refinement, and evidence-aware stopping. Using the chemistry-focused ChemKGMultiHopQA dataset, we isolate questions requiring genuine retrieval and analyze behavior with diagnostics spanning retrieval coverage gaps, anchor-carry drop, query quality, composition fidelity, and control calibration. Across models, Iterative RAG consistently outperforms Gold Context, with gains up to 25.6 percentage points, especially for non-reasoning fine-tuned models. Staged retrieval reduces late-hop failures, mitigates context overload, and enables dynamic correction of early hypothesis drift, but remaining failure modes include incomplete hop coverage, distractor latch trajectories, early stopping miscalibration, and high composition failure rates even with perfect retrieval. Overall, staged retrieval is often more influential than the mere presence of ideal evidence; we provide practical guidance for deploying and diagnosing RAG systems in specialized scientific settings and a foundation for more reliable, controllable iterative retrieval-reasoning frameworks.

Related papers

• Beyond Relevance: Bayesian Evidence Acquisition for Agentic Whole-Slide Image Reasoning [72.86819994769634]
  Whole-slide image (WSI) reasoning requires an agent to sequentially acquire visual evidence before answering a diagnostic question.<n>We propose BEACON, a plug-and-play agentic framework that reformulates WSI reasoning as a Bayesian evidence acquisition problem.
  arXiv  Detail & Related papers  (2026-08-06T08:40:57Z)
• A Systems-Level Analysis of Sensitivity, Robustness, and Stability in Retrieval-Augmented Generation [0.0]
  Retrieval-Augmented Generation (RAG) systems are often evaluated using final answer accuracy.<n>This paper presents a controlled empirical study of RAG sensitivity, robustness, and stability across 56 experimental runs.
  arXiv  Detail & Related papers  (2026-05-29T17:24:40Z)
• Skill-RAG: Failure-State-Aware Retrieval Augmentation via Hidden-State Probing and Skill Routing [12.09179106162719]
  We propose Skill-RAG, a failure-aware RAG framework that couples a hidden-state prober with a prompt-based skill router.<n> Experiments show that Skill-RAG substantially improves accuracy on hard cases persisting after multi-turn retrieval.
  arXiv  Detail & Related papers  (2026-04-17T07:25:43Z)
• Facet-Level Tracing of Evidence Uncertainty and Hallucination in RAG [8.18791900871137]
  Retrieval-Augmented Generation aims to reduce hallucinations by grounding answers in retrieved evidence.<n>Existing evaluations focus on answer-level or passage-level accuracy, offering limited insight into how evidence is used during generation.<n>We introduce a facet-level diagnostics framework for QA that decomposes each input question into atomic reasoning facets.
  arXiv  Detail & Related papers  (2026-04-10T09:59:43Z)
• Doctor-RAG: Failure-Aware Repair for Agentic Retrieval-Augmented Generation [23.961760006898157]
  Agentic Retrieval-Augmented Generation (Agentic RAG) has become a widely adopted paradigm for question answering and complex knowledge reasoning.<n>Existing approaches typically address such failures by either stopping at diagnostic analysis or rerunning the entire retrieval-reasoning pipeline.<n>We propose Doctor-RAG, a unified diagnose-and-repair framework that corrects failures in Agentic RAG through explicit error localization and prefix reuse.
  arXiv  Detail & Related papers  (2026-04-01T13:13:27Z)
• From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG [65.0203623486525]
  Large Language Models (LLMs) exhibit high reasoning capacity in medical question-answering.<n>Their tendency to produce hallucinations and outdated knowledge poses critical risks in healthcare fields.<n>Existing methods rely on noisy token-level signals and lack the multi-round refinement required for complex reasoning.
  arXiv  Detail & Related papers  (2026-02-06T08:25:30Z)
• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration [49.9937230730202]
  We propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention.<n>Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories.<n>We show that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales.
  arXiv  Detail & Related papers  (2026-02-03T15:32:09Z)
• Multi-hop Reasoning via Early Knowledge Alignment [68.28168992785896]
  Early Knowledge Alignment (EKA) aims to align Large Language Models with contextually relevant retrieved knowledge.<n>EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency.<n>EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models.
  arXiv  Detail & Related papers  (2025-12-23T08:14:44Z)
• MuISQA: Multi-Intent Retrieval-Augmented Generation for Scientific Question Answering [16.26521741515184]
  Multi-Intent Scientific Question Answering (MuISQA) benchmark designed to evaluate RAG systems on heterogeneous evidence coverage across sub-questions.<n>In addition, we propose an intent-aware retrieval framework that leverages large language models (LLMs) to hypothesize potential answers, decompose them into intent-specific queries, and retrieve supporting passages for each underlying intent.<n>Experiments on both MuISQA benchmark and other general RAG datasets demonstrate that our method consistently outperforms conventional approaches, particularly in retrieval accuracy and evidence coverage.
  arXiv  Detail & Related papers  (2025-11-20T12:03:36Z)
• Modeling Uncertainty Trends for Timely Retrieval in Dynamic RAG [35.96258615258145]
  We introduce Entropy-Trend Constraint (ETC), a training-free method that determines optimal retrieval timing by modeling the dynamics of token-level uncertainty.<n>ETC consistently outperforms strong baselines while reducing retrieval frequency.<n>It is plug-and-play, model-agnostic, and readily integrable into existing decoding pipelines.
  arXiv  Detail & Related papers  (2025-11-13T05:28:02Z)
• FAIR-RAG: Faithful Adaptive Iterative Refinement for Retrieval-Augmented Generation [0.0]
  We introduce FAIR-RAG, a novel agentic framework that transforms the standard RAG pipeline into a dynamic, evidence-driven reasoning process.<n>We conduct experiments on challenging multi-hop QA benchmarks, including HotpotQA, 2WikiMultiHopQA, and MusiQue.<n>Our work demonstrates that a structured, evidence-driven refinement process with explicit gap analysis is crucial for unlocking reliable and accurate reasoning in advanced RAG systems.
  arXiv  Detail & Related papers  (2025-10-25T15:59:33Z)
• Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
  We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
  arXiv  Detail & Related papers  (2025-10-01T07:59:03Z)
• GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis? [9.394057684388027]
  This paper introduces GALA, a novel framework for root cause analysis in microservice systems.<n> evaluated on an open-source benchmark, GALA achieves substantial improvements over state-of-the-art methods.<n>We show that GALA bridges the gap between automated failure diagnosis and practical incident resolution.
  arXiv  Detail & Related papers  (2025-08-17T19:12:05Z)
• PrismRAG: Boosting RAG Factuality with Distractor Resilience and Strategized Reasoning [57.89188317734747]
  PrismRAG trains the model with distractor-aware QA pairs mixing gold evidence with subtle distractor passages.<n>It instills reasoning-centric habits that make the LLM plan, rationalize, and synthesize without relying on extensive human engineered instructions.
  arXiv  Detail & Related papers  (2025-07-25T00:15:31Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)
• Chain-of-Retrieval Augmented Generation [91.02950964802454]
  This paper introduces an approach for training o1-like RAG models that retrieve and reason over relevant information step by step before generating the final answer.<n>Our proposed method, CoRAG, allows the model to dynamically reformulate the query based on the evolving state.
  arXiv  Detail & Related papers  (2025-01-24T09:12:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.