MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification

• URL: http://arxiv.org/abs/2609.12884v1
• Date: Fri, 11 Sep 2026 14:08:44 GMT
• Title: MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification
• Abstract summary: We introduce MedSNIP-Bench, a benchmark for snippet-level medical fact verification.<n>We also introduce MedSNIP, an automatic snippet-generation pipeline.<n>Across MedSNIP-Bench, HealthFC, and MedHallu, snippet-level verification preserves or improves false-class F1.
• Score: 50.13873960887466
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A medical claim's correctness often depends not on the claim alone, but on the clinical structure around it. A claim may require a lab reference range, a causal or conditional link, or patient-specific details to be judged correctly, and atom-level decomposition can fragment these dependencies, leaving the verifier with clinically incomplete claims. We reformulate medical fact-checking around snippet-level verification, where clause-grouped units preserve local clinical structure. We introduce MedSNIP-Bench, a human-annotated benchmark for snippet-level medical fact verification, and MedSNIP, an automatic snippet-generation pipeline. MedSNIP-Bench covers 276 consumer-health and clinical-vignette responses, segmented into 2,524 snippets with dual in-general and in-patient-context labels and six structural pattern codes. MedSNIP is evaluated against human snippet boundaries on MedSNIP-Bench and then used to generate snippet-level units for external corpora. Across MedSNIP-Bench, HealthFC, and MedHallu, snippet-level verification preserves or improves false-class F1, with gains concentrated where answers are long enough to fragment and where the verifier is strong enough to exploit the recovered structure. The largest merge-pattern gain is on causal-conditional clinical chains. It also reduces verifier calls by 24-73%, though the saving survives end-to-end only when decomposition is cheap, which an open-weight decomposer makes possible at no loss of chunking fidelity.

Related papers

• Configurable Semantic Chunking for Biomedical Information Extraction in Retrieval-Augmented Generation [0.0]
  BioMedRAG introduced retrieval-augmented generation with a learned chunk scorer for biomedical information extraction.<n>We propose a semantic chunking framework that combines entity-preserving windows, trigger-centered chunking, proposition-first extraction, tiered trigger prioritization, and hierarchical relation resolution.<n>We evaluate the framework on biomedical relation extraction benchmarks (GM-CIHT, DDI, ChemProt) and adverse event classification (ADE)
  arXiv  Detail & Related papers  (2026-08-31T17:44:54Z)
• Beyond Endpoint Gains: A Weight-Delta Audit of Medical Specialization [2.782879873824998]
  We propose a paired weight-delta path audit and apply it to two public, aligned generalist-to-medical-specialist checkpoint pairs.<n>Its claims concern text-only multiple-choice benchmark movement, not clinical validation, repair, or circuit-level mechanism.
  arXiv  Detail & Related papers  (2026-08-21T06:15:43Z)
• FedPref: Federated Preference Learning for Structured Radiology Report Extraction [69.11739400975445]
  Radiology reports describe findings and locations in free text, but downstream search and analysis require these relations in a fixed schema.<n>We introduce FedPref: frozen public language models propose alternative extractions, local annotations rank them, and sites collaboratively train pooled compact Qwen3-8B adapters.<n>On development data from six simulated hospitals with unequal data volume and disease prevalence, FedPref improves client-mean F1 by 2.49 points and worst-site F1 by 9.10 points compared with training each site in isolation.
  arXiv  Detail & Related papers  (2026-08-17T12:29:42Z)
• MedHal-Loc: Are "Explainable-by-Architecture" Medical Hallucination Detectors Faithful Localizers? A Localization Benchmark [3.738942365298833]
  We introduce MedHal-Loc, a benchmark and metric for localization faithfulness.<n>We find that NLI-per-clause, consistency-per-sentence, and the dedicated span detector FAVA all localize well above chance.
  arXiv  Detail & Related papers  (2026-06-19T15:11:37Z)
• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• CT-FineBench: A Diagnostic Fidelity Benchmark for Fine-Grained Evaluation of CT Report Generation [51.11942945171396]
  Conventional evaluation metrics offer only coarse measures of lexical overlap or entity matching.<n>We propose CT-FineBench, a benchmark built from CT-RATE and Merlin to evaluate the fine-grained factual consistency of CT reports.<n>Our benchmark is constructed through a meticulous, Question-Answering (QA) based process.
  arXiv  Detail & Related papers  (2026-04-27T03:32:46Z)
• MedVeriSeg: Teaching MLLM-Based Medical Segmentation Models to Verify Query Validity Without Extra Training [20.87443071413578]
  We propose MedVeriSeg, a training-free verification framework for LISA-like medical segmentation models.<n>Our key observation is that the similarity map between the [SEG] token feature and MLLM image features exhibits markedly different distribution patterns for true and false queries.<n>Experiments on a small-scale benchmark constructed from SA-Med2D-20M show that MedVeriSeg effectively rejects false-query segmentation requests.
  arXiv  Detail & Related papers  (2026-04-11T14:47:32Z)
• MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation [8.37586466142299]
  We introduce MedRAGChecker, a claim-level verification and diagnostic framework for biomedical RAG.<n>Given a question, retrieved evidence, and a generated answer, MedRAGChecker decomposes the answer into atomic claims and estimates claim support.<n>We show that MedRAGChecker reliably flags unsupported and contradicted claims and reveals distinct risk profiles across generators.
  arXiv  Detail & Related papers  (2026-01-10T10:40:42Z)
• MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts [70.64143198545031]
  We propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance.<n>Our results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics.
  arXiv  Detail & Related papers  (2025-10-15T12:50:33Z)
• MedScore: Generalizable Factuality Evaluation of Free-Form Medical Answers by Domain-adapted Claim Decomposition and Verification [51.82420076479152]
  We propose MedScore, a new pipeline to decompose medical answers into condition-aware valid facts and verify against in-domain corpora.<n>Our method extracts up to three times more valid facts than existing methods, reducing hallucination and vague references, and retaining condition-dependency in facts.
  arXiv  Detail & Related papers  (2025-05-24T01:23:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.