Evaluating Factual Density in Multi-Source RAG: A Study in Medical AI Accuracy

• URL: http://arxiv.org/abs/2605.31506v1
• Date: Fri, 29 May 2026 16:25:39 GMT
• Title: Evaluating Factual Density in Multi-Source RAG: A Study in Medical AI Accuracy
• Abstract summary: This paper introduces Factual Density (FD*), a novel retrieval optimization signal.<n>It measures the proportion of verified atomic claims relative to total token count.<n>FD* was the only condition to achieve 100% systematic review saturation in top-5 results.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-Augmented Generation (RAG) is the current industry standard for grounding AI in real-world facts. Traditional retrieval methods rely on keyword matching and topic proximity, ranking content based on how closely it sounds like the user's query. What they do not measure is how many verified facts the content actually contains. This structural gap, termed the Expert Blindness Effect, causes standard RAG pipelines to consistently bury high-density factual evidence in favor of lexically dominant text on the same topic. To address this gap, this paper introduces Factual Density (FD*), a novel retrieval optimization signal that measures the proportion of verified atomic claims relative to total token count. Using the NexusAgentics Ghost Audit preprocessing pipeline, raw text is scored for factual specificity using probabilistic factuality analysis to filter content before corpus ingestion. An initial formulation introduced a severe document-length confound (Pearson R = -0.8636, p = 2.27e-07). Implementing Z-score normalization within length bins resolved this bias, validating FD* as a length-independent density signal (p = 0.0749). Evaluated against the HealthFC benchmark (750 health claims labeled Supported, Refuted, or No Evidence by medical experts), FD*-optimized retrieval was the only condition to achieve 100% systematic review saturation in top-5 results, surfacing Cochrane evidence that standard cosine similarity ranked outside the top ten. Ground truth verification confirmed 25 mappings across seven HealthFC-supported claims. While full statistical validation across n=50 queries remains future work due to constraints on corpus-benchmark alignment, these findings establish factual density reranking as a low-cost, high-impact intervention for improving factual precision in health RAG architectures.

Related papers

• NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
  Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
  arXiv  Detail & Related papers  (2026-09-10T08:34:56Z)
• Recovering Clinical Utility Under Differential Privacy: Empirical Validation of Adaptive Federated Aggregation on Heterogeneous Cardiovascular Datasets [0.06372261626436676]
  The present work addresses the gap by validating the FedCVR framework on five publicly available real cardiovascular datasets.<n>Results demonstrate that FedCVR preserves its adaptive advantage on real data, achieving an F1-Score of 79.2% and AUC of 0.96.<n>The measured privacy cost on real data confirms the graceful degradation pattern observed in the synthetic experiments.
  arXiv  Detail & Related papers  (2026-07-07T18:51:13Z)
• Source-Aware Reranking for Retrieval-Augmented Generation: A Reliability Prior Approach [0.0]
  This work evaluates a simple and interpretable modification to RAG retrieval ranking that incorporates provenance-informed source reliability priors.<n>The framework is evaluated against a similarity-only baseline on a 120-document health-domain corpus.<n> source-aware reranking improves Precision@5 from 0.48 to 0.72 and reduces average adversarial document retrieval.
  arXiv  Detail & Related papers  (2026-06-08T14:09:04Z)
• Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study [0.0]
  This paper presents a systematic empirical comparison of five retrieval strategies within a biomedical question-answering RAG pipeline.<n>All strategies share a fixed generation model (GPT-4o-mini), a common vector store (ChromaDB), and OpenAI's text-em-3-small embeddings.<n> evaluation is conducted on 250 question-answer pairs drawn from a preprocessed subset of the BioASQ benchmark.
  arXiv  Detail & Related papers  (2026-05-04T12:21:46Z)
• Alignment Imprint: Zero-Shot AI-Generated Text Detection via Provable Preference Discrepancy [51.887915969023965]
  We show that modern Large Language Models (LLMs) undergo alignment, leaving a measurable distributional imprint.<n>To mitigate the instability in high-entropy regions, we introduce Log-likelihood Alignment Preference Discrepancy (LAPD)<n>LAPD is a standardized information-weighted statistic based on alignment imprint.
  arXiv  Detail & Related papers  (2026-04-18T09:12:24Z)
• Mitigating LLM Hallucinations through Domain-Grounded Tiered Retrieval [0.0]
  Large Language Models (LLMs) have achieved unprecedented fluency but remain susceptible to "hallucinations"<n>We propose a tiered retrieval and verification architecture designed to intercept factual inaccuracies by shifting LLMs from pattern-matchers to verified truth-seekers.<n>System was evaluated across 650 queries from five diverse benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T15:59:30Z)
• MedScore: Generalizable Factuality Evaluation of Free-Form Medical Answers by Domain-adapted Claim Decomposition and Verification [51.82420076479152]
  We propose MedScore, a new pipeline to decompose medical answers into condition-aware valid facts and verify against in-domain corpora.<n>Our method extracts up to three times more valid facts than existing methods, reducing hallucination and vague references, and retaining condition-dependency in facts.
  arXiv  Detail & Related papers  (2025-05-24T01:23:09Z)
• Enhancing Domain-Specific Retrieval-Augmented Generation: Synthetic Data Generation and Evaluation using Reasoning Models [0.6827423171182154]
  Retrieval-Augmented Generation (RAG) systems face significant performance gaps when applied to technical domains.<n>We propose a framework combining granular evaluation metrics with synthetic data generation to optimize domain-specific RAG performance.<n>Our empirical analysis reveals critical insights: smaller chunks (less than 10 tokens) improve precision by 31-42%.
  arXiv  Detail & Related papers  (2025-02-21T06:38:57Z)
• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• FactGenius: Combining Zero-Shot Prompting and Fuzzy Relation Mining to Improve Fact Verification with Knowledge Graphs [0.0]
  We present FactGenius, a novel method that enhances fact-checking by combining zero-shot prompting of large language models with fuzzy text matching on knowledge graphs. The evaluation of FactGenius on the FactKG, a benchmark dataset for fact verification, demonstrates that it significantly outperforms existing baselines.
  arXiv  Detail & Related papers  (2024-06-03T13:24:37Z)
• End-to-End Page-Level Assessment of Handwritten Text Recognition [69.55992406968495]
  HTR systems increasingly face the end-to-end page-level transcription of a document. Standard metrics do not take into account the inconsistencies that might appear. We propose a two-fold evaluation, where the transcription accuracy and the RO goodness are considered separately.
  arXiv  Detail & Related papers  (2023-01-14T15:43:07Z)
• Post-Contextual-Bandit Inference [57.88785630755165]
  Contextual bandit algorithms are increasingly replacing non-adaptive A/B tests in e-commerce, healthcare, and policymaking. They can both improve outcomes for study participants and increase the chance of identifying good or even best policies. To support credible inference on novel interventions at the end of the study, we still want to construct valid confidence intervals on average treatment effects, subgroup effects, or value of new policies.
  arXiv  Detail & Related papers  (2021-06-01T12:01:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.