AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents

• URL: http://arxiv.org/abs/2609.05802v1
• Date: Sat, 05 Sep 2026 01:31:50 GMT
• Title: AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents
• Abstract summary: AtomCite parses an answer into claims, checks each claim against the image of the cited page, and applies a deterministic repair policy.<n>It reaches around 93% binary verification accuracy on the injected benchmark, significantly outperforming every OCR-only condition.<n>Its repair policy lifts citation precision on the injected mix from a constructed 34% to 87-90% while retaining over 90% of correct claims.
• Score: 21.842813905857863
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models answering questions over multi-page documents are expected to cite the supporting pages, yet supplied citations are sometimes inaccurate, and current evaluations score citations at generation time or against text passages: no existing benchmark evaluates whether a system can verify and correct a page-level citation already attached to an answer. We propose AtomCite, an agentic framework that parses an answer into claims, checks each claim against the image of its cited page, and applies a deterministic repair policy. To evaluate it, we introduce DocCite, to our knowledge the first benchmark for systems that verify and correct page-level citations in document images. Built on MP-DocVQA and DUDE, it combines 928 validated injected instances with 2,468 candidate natural errors harvested from frontier- and efficiency-tier models, of which a two-annotator audit confirms 1,909 as genuine errors. Primary labels are assigned deterministically, not by LLM judges, with the human audit as a separate validation layer. Across three model families (Gemini, Claude, and GPT), AtomCite reaches around 93% binary verification accuracy on the injected benchmark, significantly outperforming every OCR-only condition, including a compute-matched control, and exceeding every prior text-based baseline given the same OCR text. Its repair policy lifts citation precision on the injected mix from a constructed 34% to 87-90% while retaining over 90% of correct claims. AtomCite also transfers: with frozen prompts and zero training, it raises the hallucination-detection scores of two open 7-8B models on five public benchmarks above the same models prompted as direct judges. Finally, the audit shows that noise in automatic labels biases measured verifier accuracy and can reverse system rankings, so evaluations relying only on synthetic or automatic labels risk mismeasuring verification capability.

Related papers

• GANDR: Claim Auditing for Verifiable Legal Answer Generation [21.842813905857863]
  Grounded-generation pipelines score the answer as a whole, so a correct conclusion can rest on fabricated or loosely matched citations and still score well.<n>We introduce GANDR, a two-agent system in which a Drafter writes an answer in a structured legal-reasoning format and a Critic, with the same view as a human verifier, audits each claim against its cited source.<n>On a 185-item legal benchmark, GANDR ranks first on every primary metric, reaching 70.8% strict accuracy and leading the strongest baseline by 11.3 points (p0.01)
  arXiv  Detail & Related papers  (2026-09-09T15:08:09Z)
• RefVerifier: Semi-Automated Reference Claim Verification for Scientific Manuscripts [1.6950215926321557]
  RefVerifier is a citation-bounded reference verification prototype designed to support academic peer review.<n>It extracts citation-bearing sentences from manuscripts, checks metadata against scholarly databases, resolves references to PDFs, localizes relevant evidence passages, and generates verdicts with natural language explanations.<n>In an end-to-end test with eight manuscripts, RefVerifier achieves a verdict accuracy of 71%.
  arXiv  Detail & Related papers  (2026-09-07T15:39:59Z)
• ATIBA: Grounded Integrity and Quality Checking for Research Papers [1.3241176321860364]
  We present ATIBA, a tool that runs five grounded integrity and quality checks on a manuscript.<n>It is designed around the same principle: an LLM is only trusted to judge, never to invent the evidence it judges against.
  arXiv  Detail & Related papers  (2026-09-03T17:24:15Z)
• HIRA: A Human-in-the-Loop Retrieval-Augmented Cascade for Document Classification in Regulated Industries [1.5965246343492343]
  HIRA is a training-free, on-premises retrieval-augmented cascade for document classification in regulated deployments.<n>Confident documents are classified directly by retrieval; uncertain or visually confusable documents are passed to a locally hosted verifier.<n>On a private 80-class trade-finance corpus, HIRA processes the full 30,233-document production stream while requesting human correction for only 1,945 documents.
  arXiv  Detail & Related papers  (2026-08-22T06:19:19Z)
• ResearchQA: Benchmarking Citation-Grounded Question-Answering on Scientific Papers [0.0]
  ResearchQA is a benchmark of 6,211 single-paper question-answer pairs from 494 open-access papers.<n>ResearchQA is designed for citation-grounded evaluation.<n>We evaluate eight leading closed- and open-weight models in a citation-grounded chat-with-paper setting.
  arXiv  Detail & Related papers  (2026-07-13T04:26:51Z)
• Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents [0.18762753243053634]
  Large language models (LLMs) power deep research agents that synthesize information from hundreds of web sources into cited reports.<n>Current approaches either trust models to self-cite accurately, risking bias, or employ retrieval-augmented generation (RAG) that does not validate source accessibility, relevance or factual consistency.<n>We introduce the first source attribution evaluation framework that uses a reproducible AST rubric to extract and evaluate inline citations at scale.
  arXiv  Detail & Related papers  (2026-05-07T17:46:45Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam [63.84155758655084]
  Humanity's Last Exam (HLE) has become a widely used benchmark for evaluating frontier large language models.<n>We introduce HLE-Verified, a verified and revised version of HLE with a transparent verification protocol and fine-grained error taxonomy.<n>We evaluate seven state-of-the-art language models on HLE and HLE-Verified, observing an average absolute accuracy gain of 7--10 percentage points.
  arXiv  Detail & Related papers  (2026-02-15T02:50:15Z)
• PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Review [54.141490756509306]
  We introduce PaperAudit-Bench, which consists of two components: PaperAudit-Dataset, an error dataset, and PaperAudit-Review, an automated review framework.<n>Experiments on PaperAudit-Bench reveal large variability in error detectability across models and detection depths.<n>We show that the dataset supports training lightweight LLM detectors via SFT and RL, enabling effective error detection at reduced computational cost.
  arXiv  Detail & Related papers  (2026-01-07T04:26:12Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs [59.836774258359945]
  OpenFactCheck is a framework for building customized automatic fact-checking systems.<n>It allows users to easily customize an automatic fact-checker and verify the factual correctness of documents and claims.<n>CheckerEVAL is a solution for gauging the reliability of automatic fact-checkers' verification results using human-annotated datasets.
  arXiv  Detail & Related papers  (2024-05-09T07:15:19Z)
• End-to-End Page-Level Assessment of Handwritten Text Recognition [69.55992406968495]
  HTR systems increasingly face the end-to-end page-level transcription of a document. Standard metrics do not take into account the inconsistencies that might appear. We propose a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately.
  arXiv  Detail & Related papers  (2023-01-14T15:43:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.