Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research

• URL: http://arxiv.org/abs/2608.24306v1
• Date: Tue, 25 Aug 2026 09:34:18 GMT
• Title: Who is the Agent to Blame? Localizing Faithfulness and Citation Mistakes in Agentic Deep Research
• Abstract summary: Deep research (DR) systems produce long-form cited reports by orchestrating multiple agents that search and synthesize information from the web.<n> Citations are the primary mechanism for evaluating the faithfulness of these reports, yet current DR systems exhibit poor citation recall.<n>We propose an evaluation method that pinpoints which agent introduced each error by locally testing agent invocations for faithfulness and verifiability relative to their own inputs.
• Score: 75.8770212601511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep research (DR) systems produce long-form cited reports by orchestrating multiple agents that search and synthesize information from the web. Citations are the primary mechanism for evaluating the faithfulness of these reports, yet current DR systems exhibit poor citation recall. Moreover, improving citation recall is challenging because DR systems are complex multi-agent architectures where information passes through agents like a telephone game, and both content and citations can get corrupted along the way. We propose an evaluation method that pinpoints which agent introduced each error by locally testing agent invocations for faithfulness and verifiability relative to their own inputs. Furthermore, we propose a four-type taxonomy to categorize the discovered errors: hallucination, uncited input reliance, uncited output, or insufficient citations. Applying our method to three top-ranked open-source DR systems, we obtain actionable diagnostics. Almost every agent makes a lot of mistakes with the exception being those that summarize a single document. We find that the dominant error type varies systematically across agents, where the orchestrator mistakes are mostly citation-related. We find that 84.7% of final-report errors in AI-Q originate at the orchestrator, roughly 31% of them hallucinations and the rest citation mistakes. Guided by these insights, we demonstrate that two simple interventions raise citation recall by 5% without degrading output quality.

Related papers

• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Writing Bug Reports for Software Repair Agents: What Information Matters Most? [45.10027008889631]
  We study what makes a bug report agent-ready.<n>We focus on the 441 issues representing bug reports.<n>Our results suggest that agentic-first reports benefit most from information that narrows the agent's search and repair space.
  arXiv  Detail & Related papers  (2026-07-10T16:00:57Z)
• What Makes a Good Bug Report for an AI Agent? [8.139494830308486]
  A good bug report for an agent overlaps with, but is not identical to, a good report for a human.<n> fix suggestions, reproduction scripts, repository source code, and localization info are associated with higher resolution likelihood.<n>Some qualities long emphasized for human readers, such as natural language steps to reproduce and readable descriptions, contribute little or even correlate with lower success.
  arXiv  Detail & Related papers  (2026-07-08T16:13:15Z)
• How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation [3.4562767039451674]
  We show how a large language model decides whether to attach an inline citation while answering a factoid question.<n>We show that amplifying or attenuating only those critical heads and eliminating 69% of spurious ones on PopQA without harming answer accuracy.<n>The results reveal a potential disconnect between the model's apparent reasoning and its internal computational pathway.
  arXiv  Detail & Related papers  (2026-06-09T07:06:13Z)
• Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories [51.22051230894794]
  Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable.<n>We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through expert review.<n>We propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path.
  arXiv  Detail & Related papers  (2026-06-01T10:50:26Z)
• BibAgent: An Agentic Framework for Traceable Miscitation Detection in Scientific Literature [21.872874595027824]
  BibAgent is a scalable, end-to-end agentic framework for automated citation verification.<n>It integrates retrieval, reasoning, and adaptive evidence aggregation, applying strategies for accessible and paywalled sources.<n>Our results demonstrate that BibAgent outperforms state-of-the-art Large Language Model (LLM) baselines in citation verification accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-01-12T16:30:45Z)
• DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence [50.97612134791782]
  Generative search engines and deep research LLM agents promise trustworthy, source-grounded synthesis, yet users regularly encounter overconfidence, weak sourcing, and confusing citation practices.<n>We introduce DeepTRACE, a novel sociotechnically grounded audit framework that turns prior community-identified failure cases into eight measurable dimensions spanning answer text, sources, and citations.
  arXiv  Detail & Related papers  (2025-09-02T00:32:38Z)
• MCP-Orchestrated Multi-Agent System for Automated Disinformation Detection [84.75972919995398]
  This paper presents a multi-agent system that uses relation extraction to detect disinformation in news articles.<n>The proposed Agentic AI system combines four agents: (i) a machine learning agent (logistic regression), (ii) a Wikipedia knowledge check agent, and (iv) a web-scraped data analyzer.<n>Results demonstrate that the multi-agent ensemble achieves 95.3% accuracy with an F1 score of 0.964, significantly outperforming individual agents and traditional approaches.
  arXiv  Detail & Related papers  (2025-08-13T19:14:48Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.