DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA

• URL: http://arxiv.org/abs/2607.14400v1
• Date: Wed, 15 Jul 2026 22:35:02 GMT
• Title: DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA
• Abstract summary: We examine a divergence between traditional natural language evaluation metrics and citation integrity as applied to RAG QA systems.<n>We evaluate a corrective pipeline using Corrective RAG (CRAG) and CiteFix against baseline and frontier model benchmark RAG QA scores.<n>We propose that evaluation of trustworthy RAG QA requires metrics that reward strict answer grounding.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper describes DS@GT ARC's submission to the CLEF 2026 LongEval Task 4 on Retrieval-Augmented Generation (RAG). In this submission, we examine a divergence between traditional natural language evaluation metrics and citation integrity as applied to RAG QA systems. We evaluate a corrective pipeline using Corrective RAG (CRAG) and CiteFix against baseline and frontier model benchmark RAG QA scores. While frontier models maximized answer relevance and fluency scores, our RAGAs LLM-as-judge diagnostics indicate that frontier models would correctly identify relevant documents without using their context in answer generation. Conversely, by filtering chunks pre-generation and enforcing strict entailment of generated claims to the cited material post-generation, our corrective pipeline marginally improved citation faithfulness and answer grounding. We propose that evaluation of trustworthy RAG QA requires metrics that reward strict answer grounding.

Related papers

• Towards Query-Agnostic RAG Evaluation via Query Coverage and Claim Verifiability [29.6470498810458]
  We propose Q-CARE, a query-agnostic and fully reference-free framework for fine-grained assessment.<n>Q-CARE decomposes queries into sub-queries and answers into atomic claims.<n>On a human-annotated benchmark, Q-CARE achieves higher correlation with human judgments than four existing RAG evaluation metrics.
  arXiv  Detail & Related papers  (2026-07-31T09:21:43Z)
• MIRAGE: Defending Long-Form RAG Against Misinformation Pollution [87.40537931342082]
  We introduce MIRAGE, a training-free, model-agnostic defense for long-form RAG.<n>MIRAGE builds an NLI-based cross-document claim graph and applies a Defended-Claims Gate to either condition generation on a consistent, multi-source supported subset or to block retrieval and answer parametrically.
  arXiv  Detail & Related papers  (2026-07-06T13:36:12Z)
• Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering [54.97384993676565]
  We formalize Regulatory Compliance QA with RegOps-Bench, a novel benchmark featuring an Operational Knowledge Graph derived from complex national R&D regulations.<n>RefWalk traverses cross-document citations, fuses multi-view candidates via max-based aggregation, and enforces per-rule attribution to explicitly map claims to sources.
  arXiv  Detail & Related papers  (2026-05-28T10:38:38Z)
• ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering [54.72902502486611]
  ReAG is a Reasoning-Augmented Multimodal RAG approach that combines coarse- and fine-grained retrieval with a critic model that filters irrelevant passages.<n>ReAG significantly outperforms prior methods, improving answer accuracy and providing interpretable reasoning grounded in retrieved evidence.
  arXiv  Detail & Related papers  (2025-11-27T19:01:02Z)
• Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality [6.970520292379132]
  Conformal-RAG is a novel framework inspired by recent applications of conformal prediction (CP) on large language models (LLMs)<n>It offers statistical guarantees on the quality of refined sub-claims, ensuring response reliability without the need for ground truth answers.
  arXiv  Detail & Related papers  (2025-06-26T03:52:56Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)
• Unanswerability Evaluation for Retrieval Augmented Generation [74.3022365715597]
  UAEval4RAG is a framework designed to evaluate whether RAG systems can handle unanswerable queries effectively.<n>We define a taxonomy with six unanswerable categories, and UAEval4RAG automatically synthesizes diverse and challenging queries.
  arXiv  Detail & Related papers  (2024-12-16T19:11:55Z)
• RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answering [18.59602224175808]
  We propose Rule-guided Retrieval-Augmented Generation with LMs, which explicitly introduces rules for in-context learning (RuleRAG-ICL)<n> Experiments on RuleQA demonstrate RuleRAG-ICL improves the retrieval quality of +89.2% in Recall@10 and answer accuracy of +103.1% in Exact Match.<n>In addition, experiments on four existing RAG datasets show RuleRAG is also effective by offering rules in RuleQA to them.
  arXiv  Detail & Related papers  (2024-10-15T14:51:45Z)
• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework [66.93260816493553]
  This paper introduces RAGEval, a framework designed to assess RAG systems across diverse scenarios.<n>With a focus on factual accuracy, we propose three novel metrics: Completeness, Hallucination, and Irrelevance.<n> Experimental results show that RAGEval outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.
  arXiv  Detail & Related papers  (2024-08-02T13:35:11Z)
• Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation [51.8188846284153]
  Attributed Text Generation (ATG) is proposed to enhance credibility and verifiability in RAG systems.<n>This paper proposes ReClaim, a fine-grained ATG method that alternates the generation of references and answers step by step.<n>With extensive experiments, we verify the effectiveness of ReClaim in extensive settings, achieving a citation accuracy rate of 90%.
  arXiv  Detail & Related papers  (2024-07-01T20:47:47Z)
• Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation [8.975024781390077]
  We present MIRAGE --Model Internals-based RAG Explanations -- a plug-and-play approach using model internals for faithful answer attribution in question answering applications. We evaluate our proposed approach on a multilingual QA dataset, finding high agreement with human answer attribution.
  arXiv  Detail & Related papers  (2024-06-19T16:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.