Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG

• URL: http://arxiv.org/abs/2608.02011v2
• Date: Tue, 04 Aug 2026 14:04:12 GMT
• Title: Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG
• Abstract summary: Agentic retrieval-augmented generation (RAG) systems can fail before evidence-conditioned reasoning is tested.<n>We study this failure mode as a procedural property of the agent trajectory.<n>We evaluate ReadGate, a minimal runtime requiring an agent to read after search and before finalization.
• Score: 0.7161783472741748
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Agentic retrieval-augmented generation (RAG) systems can fail before evidence-conditioned reasoning is tested: an agent may retrieve candidate snippets but finalize without inspecting them. We study this failure mode as a procedural property of the agent trajectory, decomposing wrong answers into pre-evidence discipline failures and post-gold-read failures using saved tool-call traces, retrieved evidence, read passages, and final answers. Across 12,000 paired trajectories on HotpotQA, 2WikiMultiHopQA, and MuSiQue, the two failure types are largely non-redundant: the both-trigger rate is in [11.2%, 13.1%] across regex and spaCy entity extractors. We then evaluate Read-Gate, a minimal runtime invariant requiring an agent to read after search and before finalization. Forced reading improves LLM-Acc by 14.9-19.9 points on trajectories that would otherwise skip reading and by 3.2-9.4 points on full minimal-reasoning cells. Additional diagnostics show that larger hidden thinking budgets do not necessarily increase evidence inspection. Together, these results indicate that evidence-gathering should be evaluated as a trajectory-level control problem, separately from answer-side reasoning.

Related papers

• When Collaboration Becomes a Trigger: Collective Evidence-Threshold Backdoors in Multi-Agent Systems [57.450132281381066]
  Backdoor behavior can be activated when peer evidence reaches a hidden threshold, rather than being determined by any single message.<n>We introduce a collective evidence-threshold backdoor paradigm for MAS and Boundary-Conditioned Backdoor Injection (BCBI)<n>We propose LAtent Transition Test-time Evaluation (LATTE), a clean-only latent-transition defense that learns benign communication dynamics and quarantines anomalous agent updates.
  arXiv  Detail & Related papers  (2026-08-02T08:38:44Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• When RLVR Shrinks the Reasoning Boundary: Diagnosing Pass@k Inversion [0.0]
  Reinforcement learning with verifiable rewards (RLVR) can improve one-sample accuracy while making a model worse under repeated sampling.<n>We study this pass@k inversion: after training, the policy may solve fewer distinct problems than its base model at large $k$.<n>The failure concentrates on boundary prompts, where the base model contains rare correct trajectories that are recoverable by sampling but too sparse to reliably appear in finite RLVR rollout groups.
  arXiv  Detail & Related papers  (2026-07-12T17:17:18Z)
• When Evidence is Sparse: Weakly Supervised Early Failure Alerting in Dialogs and LLM-Agent Trajectories [20.332123003765837]
  Early failure alerting requires deciding, while a dialog or agent trajectory is still unfolding, whether to flag it as likely to fail.<n>This is challenging because supervision is typically available only as a trajectory-level success/failure label while alerts must be raised from partial interactions.<n>We introduce a two-stage approach that learns from this sparse evidence structure and uses the resulting risk estimates for controllable early alerting.
  arXiv  Detail & Related papers  (2026-06-03T20:28:27Z)
• FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search [11.638321375070047]
  We propose FALAT, a diagnostic framework for failure attribution in LLM agent trajectories.<n>We evaluate FALAT on the Who&When benchmark, which includes both algorithm-generated and hand-crafted multi-agent failure trajectories.
  arXiv  Detail & Related papers  (2026-05-30T15:11:35Z)
• ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling [57.42714978834704]
  ExComm is a communication protocol for exploration-stage agentic test-time scaling.<n>We show that ExComm consistently outperforms strong test-time scaling baselines.
  arXiv  Detail & Related papers  (2026-05-21T07:38:44Z)
• VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems [79.51005192758262]
  Large language model-driven multi-agent systems excel at complex tasks.<n>However, unreliable agents remain a key bottleneck to system-level reliability.<n>We propose a hypothesis verification framework for agent failure attribution.
  arXiv  Detail & Related papers  (2026-05-17T14:09:35Z)
• Affordance Agent Harness: Verification-Gated Skill Orchestration [45.231685718099264]
  Affordance grounding requires identifying where and how an agent should interact in open-world scenes.<n>We propose Affordance Agent Harness, a closed-loop runtime that unifies skills with an evidence store and cost control.
  arXiv  Detail & Related papers  (2026-05-01T13:45:16Z)
• PAR$^2$-RAG: Planned Active Retrieval and Reasoning for Multi-Hop Question Answering [57.89576196160413]
  Large language models (LLMs) remain brittle on multi-hop question answering (MHQA)<n>We propose textbfPlanned Active Retrieval and Reasoning RAG (PAR$2$-RAG), a two-stage framework that separates emphcoverage from emphcommitment.
  arXiv  Detail & Related papers  (2026-03-30T23:52:54Z)
• AEGIS: From Clues to Verdicts -- Graph-Guided Deep Vulnerability Reasoning via Dialectics and Meta-Auditing [9.271196825503417]
  Large Language Models (LLMs) are increasingly adopted for vulnerability detection, yet their reasoning remains fundamentally unsound.<n>We present AEGIS, a novel multi-agent framework that shifts detection from ungrounded speculation to forensic verification over a closed factual substrate.<n>It reduces the false positive rate by up to 54.40% compared to leading baselines, at an average cost of $0.09 per sample without any task-specific training.
  arXiv  Detail & Related papers  (2026-03-21T04:12:04Z)
• Thinking Traps in Long Chain-of-Thought: A Measurable Study and Trap-Aware Adaptive Restart [27.904791075662896]
  We introduce TAAR (Trap-Aware Adaptive Restart), a test-time control framework that trains a diagnostic policy to predict two signals from partial trajectories.<n>At inference time, TAAR truncates the trajectory before the predicted trap segment and adaptively restarts decoding.<n>Experiments show that TAAR improves reasoning performance without fine-tuning base model parameters.
  arXiv  Detail & Related papers  (2026-01-17T07:26:02Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction [58.51530390018909]
  Large Language Model based multi-agent systems excel at collaborative problem solving but remain brittle to cascading errors.<n>We present MASC, a metacognitive framework that endows MAS with real-time, unsupervised, step-level error detection and self-correction.
  arXiv  Detail & Related papers  (2025-10-16T05:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.