Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents

• URL: http://arxiv.org/abs/2605.06635v1
• Date: Thu, 07 May 2026 17:46:45 GMT
• Title: Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents
• Abstract summary: Large language models (LLMs) power deep research agents that synthesize information from hundreds of web sources into cited reports.<n>Current approaches either trust models to self-cite accurately, risking bias, or employ retrieval-augmented generation (RAG) that does not validate source accessibility, relevance or factual consistency.<n>We introduce the first source attribution evaluation framework that uses a reproducible AST rubric to extract and evaluate inline citations at scale.
• Score: 0.18762753243053634
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) power deep research agents that synthesize information from hundreds of web sources into cited reports, yet these citations cannot be reliably verified. Current approaches either trust models to self-cite accurately, risking bias, or employ retrieval-augmented generation (RAG) that does not validate source accessibility, relevance, or factual consistency. We introduce the first source attribution evaluation framework that uses a reproducible AST parser to extract and evaluate inline citations from LLM-generated Markdown reports at scale. Unlike methods that verify claims in isolation, our framework closes the loop by retrieving the actual cited content, enabling human or model evaluators to judge each citation against its source. Citations are evaluated along three dimensions. (1) Link Works verifies URL accessibility, (2) Relevant Content measures topical alignment, and (3) Fact Check validates factual accuracy against source content. We benchmark 14 closed-source and open-source LLMs across three evaluation dimensions using rubric-based LLM-as-a-judge evaluators calibrated through human review. Our results reveal that even the strongest frontier models maintain link validity above 94% and relevance above 80%, yet achieve only 39-77% factual accuracy, while fewer than half of open-source models successfully generate cited reports in a one-shot setting. Ablation studies on research depth show that Fact Check accuracy drops by approximately 42% on average across two frontier models as tool calls scale from 2 to 150, demonstrating that more retrieval does not produce more accurate citations. These findings reveal a critical disconnect between surface-level citation quality and factual reliability, and our framework provides the evaluation infrastructure to assess the disconnect.

Related papers

• How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation [3.4562767039451674]
  We show how a large language model decides whether to attach an inline citation while answering a factoid question.<n>We show that amplifying or attenuating only those critical heads and eliminating 69% of spurious ones on PopQA without harming answer accuracy.<n>The results reveal a potential disconnect between the model's apparent reasoning and its internal computational pathway.
  arXiv  Detail & Related papers  (2026-06-09T07:06:13Z)
• Verified Misguidance: Measuring Structural Citation Failures in Search-Augmented LLMs [8.721564756242431]
  CITETRACE is a dataset that traces the full citation chain from user query through retrieved source to generated answer.<n>We design a three-dimension evaluation framework that scores each citation on intent-purpose alignment, source suitability, and answer-source fidelity.<n>Across our pool, 30.6% of citations distort their sources and 27.1% originate from domain-inappropriate sources.
  arXiv  Detail & Related papers  (2026-05-27T14:54:05Z)
• CRISP: Characterizing Relative Impact of Scholarly Publications [70.25915589244961]
  We propose CRISP, which jointly ranks all cited papers within a citing paper using large language models (LLMs)<n>To mitigate LLMs' positional bias, we rank each list three times in a randomized order and aggregate the impact labels through majority voting.<n>CRISP outperforms a prior state-of-the-art impact classifier by +9.5% accuracy and +8.3% F1 on a dataset of human-annotated citations.
  arXiv  Detail & Related papers  (2026-03-25T16:42:30Z)
• Leveraging LLM Parametric Knowledge for Fact Checking without Retrieval [60.25608870901428]
  Trustworthiness is a core research challenge for agentic AI systems built on Large Language Models (LLMs)<n>We propose the task of fact-checking without retrieval, focusing on the verification of arbitrary natural language claims, independent of their source robustness.
  arXiv  Detail & Related papers  (2026-03-05T18:42:51Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• GhostCite: A Large-Scale Analysis of Citation Validity in the Age of Large Language Models [22.147294042024836]
  Citations provide the basis for trusting scientific claims; when they are invalid or fabricated, this trust collapses.<n>With the advent of Large Language Models (LLMs), this risk has intensified.<n>We develop CiteVerifier, an open-source framework for large-scale citation verification.
  arXiv  Detail & Related papers  (2026-02-06T14:08:34Z)
• ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks [14.371010711040304]
  ReportBench is a benchmark designed to evaluate the content quality of research reports generated by large language models (LLMs)<n>Our evaluation focuses on two critical dimensions: (1) the quality and relevance of cited literature, and (2) the faithfulness and veracity of the statements within the generated reports.
  arXiv  Detail & Related papers  (2025-08-14T03:33:43Z)
• CiteEval: Principle-Driven Citation Evaluation for Source Attribution [38.24323805177938]
  CiteEval is a citation evaluation framework driven by principles focusing on fine-grained citation assessment.<n>CiteBench is a benchmark with high-quality human annotations on citation quality.<n>CiteEval-Auto is a suite of model-based metrics that exhibit strong correlation with human judgments.
  arXiv  Detail & Related papers  (2025-06-02T16:15:34Z)
• Document Attribution: Examining Citation Relationships using Large Language Models [62.46146670035751]
  We propose a zero-shot approach that frames attribution as a straightforward textual entailment task.<n>We also explore the role of the attention mechanism in enhancing the attribution process.
  arXiv  Detail & Related papers  (2025-05-09T04:40:11Z)
• CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation [76.31621715032558]
  Grounded generation aims to equip language models (LMs) with the ability to produce more credible and accountable responses. We introduce CaLM, a novel verification framework. Our framework empowers smaller LMs, which rely less on parametric memory, to validate the output of larger LMs.
  arXiv  Detail & Related papers  (2024-06-08T06:04:55Z)
• LLMs Can Patch Up Missing Relevance Judgments in Evaluation [56.51461892988846]
  We use large language models (LLMs) to automatically label unjudged documents. We simulate scenarios with varying degrees of holes by randomly dropping relevant documents from the relevance judgment in TREC DL tracks. Our method achieves a Kendall tau correlation of 0.87 and 0.92 on an average for Vicuna-7B and GPT-3.5 Turbo respectively.
  arXiv  Detail & Related papers  (2024-05-08T00:32:19Z)
• Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data [48.409306245463]
  We develop models that quote verbatim statements from trusted sources in their pre-training data.<n>The core of Quote-Tuning is a fast membership inference function that efficiently verifies text against trusted corpora.<n> Experiments show that Quote-Tuning significantly increases verbatim quotes from high-quality documents by up to 130% relative to base models.
  arXiv  Detail & Related papers  (2024-04-05T02:27:09Z)
• Trusted Source Alignment in Large Language Models [30.14375102262399]
  We present FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking articles. We find that as we scale up the model size, the model performance on FactCheckQA improves from near-random to up to 80% balanced accuracy in aligning with trusted sources.
  arXiv  Detail & Related papers  (2023-11-12T00:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.