FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment

• URL: http://arxiv.org/abs/2603.19539v1
• Date: Fri, 20 Mar 2026 00:33:58 GMT
• Title: FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment
• Abstract summary: We introduce an expert curated, real-world benchmark for evaluating document-grounded question-answering (QA) motivated by generic drug assessment.<n>Drug labels contain rich but heterogeneous clinical and regulatory information.<n>FDARxBench generates high-quality, expert curated, QA examples spanning factual, multi-hop, and refusal tasks.
• Score: 28.49642871832232
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce an expert curated, real-world benchmark for evaluating document-grounded question-answering (QA) motivated by generic drug assessment, using the U.S. Food and Drug Administration (FDA) drug label documents. Drug labels contain rich but heterogeneous clinical and regulatory information, making accurate question answering difficult for current language models. In collaboration with FDA regulatory assessors, we introduce FDARxBench, and construct a multi-stage pipeline for generating high-quality, expert curated, QA examples spanning factual, multi-hop, and refusal tasks, and design evaluation protocols to assess both open-book and closed-book reasoning. Experiments across proprietary and open-weight models reveal substantial gaps in factual grounding, long-context retrieval, and safe refusal behavior. While motivated by FDA generic drug assessment needs, this benchmark also provides a substantial foundation for challenging regulatory-grade evaluation of label comprehension. The benchmark is designed to support evaluation of LLM behavior on drug-label questions.

Related papers

• DrugReason: Dynamic Multi-View Reasoning over Knowledge Graph and Language Evidence for Drug Repurposing [55.45112712244231]
  DrugReason is a multi-view reasoning framework that integrates grounded KG reasoning with LLM-generated mechanistic inference for drug repurposing.<n> Experiments on PharmaDB, DDInter, and DrugBank show that DrugReason improves average performance over strong single-view reasoning baselines.
  arXiv  Detail & Related papers  (2026-09-06T18:54:26Z)
• Can I Take Another Dose? Evaluating LLM Decision-Making Under Temporal Uncertainty in OTC Dosing QA [1.3583317564926913]
  We introduce DOSEBENCH, a focused benchmark of 81 curated OTC dosing scenarios.<n>We evaluate four LLMs across repeated runs using metrics for decision correctness, consistency, explanation verifiability, failure types, and confidence-related signals.
  arXiv  Detail & Related papers  (2026-06-02T22:30:15Z)
• Citation-Closure Retrieval and Per-Rule Attribution for Real-World Regulatory Compliance Question Answering [54.97384993676565]
  We formalize Regulatory Compliance QA with RegOps-Bench, a novel benchmark featuring an Operational Knowledge Graph derived from complex national R&D regulations.<n>RefWalk traverses cross-document citations, fuses multi-view candidates via max-based aggregation, and enforces per-rule attribution to explicitly map claims to sources.
  arXiv  Detail & Related papers  (2026-05-28T10:38:38Z)
• MedDCR: Learning to Design Agentic Workflows for Medical Coding [55.51674334874892]
  Medical coding converts free-text clinical notes into standardized diagnostic and procedural codes.<n>We present MedDCR, a closed-loop framework that treats design as a learning problem.<n>On benchmark datasets, MedDCR outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-11-17T13:30:51Z)
• OIDA-QA: A Multimodal Benchmark for Analyzing the Opioid Industry Documents Archive [50.468138755368805]
  Opioid crisis represents a significant moment in public health.<n>Data and documents disclosed in the UCSF-JHU Opioid Industry Documents Archive (OIDA)<n>In this paper, we tackle this challenge by organizing the original dataset according to document attributes.
  arXiv  Detail & Related papers  (2025-11-13T03:27:32Z)
• DRBench: A Realistic Benchmark for Enterprise Deep Research [81.49694432639406]
  DRBench is a benchmark for evaluating AI agents on complex, open-ended deep research tasks in enterprise settings.<n>We release 15 deep research tasks across 10 domains, such as Sales, Cybersecurity, and Compliance.
  arXiv  Detail & Related papers  (2025-09-30T18:47:20Z)
• From Feedback to Checklists: Grounded Evaluation of AI-Generated Clinical Notes [18.88487525876338]
  We propose a pipeline that distills real user feedback into structured checklists for note evaluation.<n>Using deidentified data from over 21,000 clinical encounters, we show that our feedback-derived checklist outperforms a baseline approach in offline evaluations.<n>In offline research settings, our checklist offers a practical tool for flagging notes that may fall short of our defined quality standards.
  arXiv  Detail & Related papers  (2025-07-23T17:28:31Z)
• Standard Applicability Judgment and Cross-jurisdictional Reasoning: A RAG-based Framework for Medical Device Compliance [3.439579933384111]
  Given a free-text device description, our system retrieves candidate standards from a curated corpus and uses large language models to infer jurisdiction-specific applicability.<n>We construct an international benchmark dataset of medical device descriptions with expert-annotated standard mappings, and evaluate our system against retrieval-only, zero-shot, and rule-based baselines.<n>The proposed approach attains a classification accuracy of 73% and a Top-5 retrieval recall of 87%, demonstrating its effectiveness in identifying relevant regulatory standards.
  arXiv  Detail & Related papers  (2025-06-23T11:04:58Z)
• medicX-KG: A Knowledge Graph for Pharmacists' Drug Information Needs [0.0]
  Leveraging artificial intelligence and semantic technologies, Knowledge Graphs (KGs) uncover hidden relationships and enable data-driven decision-making.<n>This paper presents medicX-KG, a pharmacist-oriented knowledge graph supporting clinical and regulatory decisions.<n>It forms the semantic layer of the broader medicX platform, powering predictive and explainable pharmacy services.
  arXiv  Detail & Related papers  (2025-06-22T09:28:48Z)
• CRAB: A Benchmark for Evaluating Curation of Retrieval-Augmented LLMs in Biomedicine [3.1427813443719868]
  We introduce the benchmark for Curation of Retrieval-Augmented LLMs in Biomedicine (CRAB)<n>CRAB is the first multilingual benchmark tailored for evaluating the biomedical curation of retrieval-augmented LLMs.<n>By incorporating a novel citation-based evaluation metric, CRAB quantifies the curation performance of retrieval-augmented LLMs in biomedicine.
  arXiv  Detail & Related papers  (2025-04-15T07:25:40Z)
• Comprehensive and Practical Evaluation of Retrieval-Augmented Generation Systems for Medical Question Answering [70.44269982045415]
  Retrieval-augmented generation (RAG) has emerged as a promising approach to enhance the performance of large language models (LLMs) We introduce Medical Retrieval-Augmented Generation Benchmark (MedRGB) that provides various supplementary elements to four medical QA datasets. Our experimental results reveals current models' limited ability to handle noise and misinformation in the retrieved documents.
  arXiv  Detail & Related papers  (2024-11-14T06:19:18Z)
• Knowledge-Driven New Drug Recommendation [88.35607943144261]
  We develop a drug-dependent multi-phenotype few-shot learner to bridge the gap between existing and new drugs. EDGE eliminates the false-negative supervision signal using an external drug-disease knowledge base. Results show that EDGE achieves 7.3% improvement on the ROC-AUC score over the best baseline.
  arXiv  Detail & Related papers  (2022-10-11T16:07:52Z)
• Principal-Agent Hypothesis Testing [54.154244569974864]
  We consider the relationship between a regulator (the principal) and an experimenter (the agent) such as a pharmaceutical company. The efficacy of the drug is not known to the regulator, so the pharmaceutical company must run a costly trial to prove efficacy to the regulator. We show how to design protocols that are robust to an agent's strategic actions, and derive the optimal protocol in the presence of strategic entrants.
  arXiv  Detail & Related papers  (2022-05-13T17:59:23Z)
• Large-Scale Data Mining of Rapid Residue Detection Assay Data From HTML and PDF Documents: Improving Data Access and Visualization for Veterinarians [3.055086390437118]
  Extra-label drug use in food animal medicine is authorized by the US Animal Medicinal Drug Use Clarification Act (AMDUCA) Occasionally there is a paucity of scientific data on which to base a withdrawal interval or a large number of animals being treated, driving the need to test for drug residues. Rapid assay commercial farm-side tests are essential for monitoring drug residues in animal products to protect human health.
  arXiv  Detail & Related papers  (2021-12-02T03:39:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.