Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution

• URL: http://arxiv.org/abs/2603.05308v1
• Date: Thu, 05 Mar 2026 15:48:43 GMT
• Title: Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution
• Abstract summary: We present Med-V1, a family of small language models with only three billion parameters.<n>We train Med-V1 on high-quality synthetic data newly developed in this study.
• Score: 41.35122660643713
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Assessing whether an article supports an assertion is essential for hallucination detection and claim verification. While large language models (LLMs) have the potential to automate this task, achieving strong performance requires frontier models such as GPT-5 that are prohibitively expensive to deploy at scale. To efficiently perform biomedical evidence attribution, we present Med-V1, a family of small language models with only three billion parameters. Trained on high-quality synthetic data newly developed in this study, Med-V1 substantially outperforms (+27.0% to +71.3%) its base models on five biomedical benchmarks unified into a verification format. Despite its smaller size, Med-V1 performs comparably to frontier LLMs such as GPT-5, along with high-quality explanations for its predictions. We use Med-V1 to conduct a first-of-its-kind use case study that quantifies hallucinations in LLM-generated answers under different citation instructions. Results show that the format instruction strongly affects citation validity and hallucination, with GPT-5 generating more claims but exhibiting hallucination rates similar to GPT-4o. Additionally, we present a second use case showing that Med-V1 can automatically identify high-stakes evidence misattributions in clinical practice guidelines, revealing potentially negative public health impacts that are otherwise challenging to identify at scale. Overall, Med-V1 provides an efficient and accurate lightweight alternative to frontier LLMs for practical and real-world applications in biomedical evidence attribution and verification tasks. Med-V1 is available at https://github.com/ncbi-nlp/Med-V1.

Related papers

• Do No Harm? Hallucination and Actor-Level Abuse in Web-Deployed Medical Large Language Models [1.2326533234282089]
  Web-facing medical large language models (LLMs) pose risks of hallucination, policy noncompliance, and unsafe design.<n>We conduct a large-scale assessment of 6,233 MedGPTs, evaluating a stratified sample of 1,500, together with 10 open-source LLMs.<n>Results show that 25-30% of MedGPTs exhibit low factual accuracy, with bottom- and middle-tier models at highest risk.
  arXiv  Detail & Related papers  (2026-05-20T00:57:59Z)
• Evaluating Small Open LLMs for Medical Question Answering: A Practical Framework [0.0]
  Large language models (LLMs) in medical question answering demands more than high average accuracy.<n>Online health communities such as Reddit have become a primary source of medical information for millions of users.<n>We present a practical, open-source evaluation framework for assessing small, locally-deployable open-weight LLMs on medical question answering.
  arXiv  Detail & Related papers  (2026-04-12T08:56:15Z)
• Simulating Clinical AI Assistance using Multimodal LLMs: A Case Study in Diabetic Retinopathy [0.0]
  Diabetic retinopathy (DR) is a leading cause of blindness worldwide, and AI systems can expand access to fundus photography screening.<n>We evaluated large language models (MLLMs) for DR and their ability to simulate clinical AI assistance across different output types.<n>These findings suggest MLLMs may improve DR screening pipelines and serve as scalable simulators for studying clinical AI assistance across varying output configurations.
  arXiv  Detail & Related papers  (2025-09-16T16:42:19Z)
• MedGemma Technical Report [75.88152277443179]
  We introduce MedGemma, a collection of medical vision-language foundation models based on Gemma 3 4B and 27B.<n>MedGemma demonstrates advanced medical understanding and reasoning on images and text.<n>We additionally introduce MedSigLIP, a medically-tuned vision encoder derived from SigLIP.
  arXiv  Detail & Related papers  (2025-07-07T17:01:44Z)
• Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content [0.10241134756773229]
  We introduce Biomed-Enriched, a biomedical text dataset constructed from PubMed via a two-stage annotation process.<n>In the first stage, a large language model annotates 400K paragraphs from PubMed scientific articles, assigning scores for their type (review, study, clinical case, other), domain (clinical, biomedical, other), and educational quality.<n>The resulting metadata allows us to extract refined subsets, including 2M clinical case paragraphs with over 450K high-quality ones from articles with commercial-use licenses.
  arXiv  Detail & Related papers  (2025-06-25T11:30:25Z)
• MedHal: An Evaluation Dataset for Medical Hallucination Detection [4.98142540436183]
  We present MedHal, a novel large-scale dataset specifically designed to evaluate if models can detect hallucinations in medical texts.<n>MedHal addresses gaps by: (1) incorporating diverse medical text sources and tasks; (2) providing a substantial volume of annotated samples suitable for training medical hallucination detection models; and (3) including explanations for factual inconsistencies to guide model learning.
  arXiv  Detail & Related papers  (2025-04-11T14:55:15Z)
• MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models [81.64135119165277]
  hallucinations can jeopardize clinical decision making, potentially harming the diagnosis and treatments.<n>We propose MedHallTune, a large-scale benchmark designed specifically to evaluate and mitigate hallucinations in medical VLMs.<n>We conduct a comprehensive evaluation of current medical and general VLMs using MedHallTune, assessing their performance across key metrics, including clinical accuracy, relevance, detail level, and risk level.
  arXiv  Detail & Related papers  (2025-02-28T06:59:49Z)
• MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning [29.84956540178252]
  Reasoning is a critical frontier for advancing medical image analysis.<n>We introduce MedVLM-R1, a medical VLM that explicitly generates natural language reasoning.<n>MedVLM-R1 boosts accuracy from 55.11% to 78.22% across MRI, CT, and X-ray benchmarks.
  arXiv  Detail & Related papers  (2025-02-26T23:57:34Z)
• MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models [82.30696225661615]
  We introduce MedHallu, the first benchmark specifically designed for medical hallucination detection.<n>We show that state-of-the-art LLMs, including GPT-4o, Llama-3.1, and medically fine-tuned UltraMedical, struggle with this binary hallucination detection task.<n>Using bidirectional entailment clustering, we show that harder-to-detect hallucinations are semantically closer to ground truth.
  arXiv  Detail & Related papers  (2025-02-20T06:33:23Z)
• Capabilities of Gemini Models in Medicine [100.60391771032887]
  We introduce Med-Gemini, a family of highly capable multimodal models specialized in medicine. We evaluate Med-Gemini on 14 medical benchmarks, establishing new state-of-the-art (SoTA) performance on 10 of them. Our results offer compelling evidence for Med-Gemini's potential, although further rigorous evaluation will be crucial before real-world deployment.
  arXiv  Detail & Related papers  (2024-04-29T04:11:28Z)
• Customizing General-Purpose Foundation Models for Medical Report Generation [64.31265734687182]
  The scarcity of labelled medical image-report pairs presents great challenges in the development of deep and large-scale neural networks. We propose customizing off-the-shelf general-purpose large-scale pre-trained models, i.e., foundation models (FMs) in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-06-09T03:02:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.