DrugClaw and DrugAudit: A Primary-Source-Grounded Agent and Authority-Aware Benchmark for Drug-Information Question Answering

• URL: http://arxiv.org/abs/2606.01434v1
• Date: Sun, 31 May 2026 20:11:05 GMT
• Title: DrugClaw and DrugAudit: A Primary-Source-Grounded Agent and Authority-Aware Benchmark for Drug-Information Question Answering
• Abstract summary: We present DrugClaw, a multi-agent retrieval-augmented system that queries a registry of drug and pharmacovigilance skills.<n>We also contribute DrugAudit, a 3,772-item authority-aware benchmark.<n>Across DrugAudit plus drug-related subsets of MedQA (751) and PubMedQA, DrugClaw is top-1 on every column of the headline table.
• Score: 21.327464065781694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Drug-information question answering is a high-stakes setting where hallucinated facts can mislead clinical decision-making and the provenance of each cited fact matters as much as the fact itself. We present DrugClaw, a multi-agent retrieval-augmented system that queries a registry of drug and pharmacovigilance skills via a reflection-driven state-machine workflow and returns answers grounded in primary regulatory or peer-reviewed records. We also contribute DrugAudit, a 3,772-item authority-aware benchmark with an evaluation panel that scores upstream-of-gold source match, token-level semantic snippet overlap, and citation faithfulness under a dual-judge LLM-as-judge protocol with inter-judge kappa = 0.88 (almost-perfect). Across DrugAudit plus drug-related subsets of MedQA (751) and PubMedQA (512), DrugClaw is top-1 on every column of the headline table: composite Evidence Index under both judges, judge-mediated answer correctness, primary-source rate (0.918, +10.1 pp over next-best), faithfulness (0.887, +5.9 pp), MedQA (0.920), and PubMedQA (0.693).

Related papers

• Curation and Extraction of Drug-Related Entities from Reddit Platform [12.718020016122908]
  We introduce ReDose, a dataset of 6,435 Reddit posts on substance use.<n>A board-certified toxicologist primarily annotated both the training and test sets.<n>We benchmarked 6,267 annotations using BERT-based, large language model (LLM)-based, and Retrieval-Augmented Generation (RAG) models.
  arXiv  Detail & Related papers  (2026-04-08T05:05:52Z)
• FDARxBench: Benchmarking Regulatory and Clinical Reasoning on FDA Generic Drug Assessment [28.49642871832232]
  We introduce an expert curated, real-world benchmark for evaluating document-grounded question-answering (QA) motivated by generic drug assessment.<n>Drug labels contain rich but heterogeneous clinical and regulatory information.<n>FDARxBench generates high-quality, expert curated, QA examples spanning factual, multi-hop, and refusal tasks.
  arXiv  Detail & Related papers  (2026-03-20T00:33:58Z)
• From Archives to Decisions: Multi-Agent Pharmaceutical Co-Scientist for Traceable Drug Discovery and Reverse Translation [0.6347968479268647]
  We introduce DiscoVerse, a multi-agent co-scientist designed to support pharmaceutical research and development.<n>The system implements semantic retrieval, cross-document linking, and auditable synthesis on a large historical corpus from Roche.<n>To validate our approach at real-world scale, we selected a subset of 180 molecules from the Roche research repositories, covering over 0.87 billion BPE tokens and more than four decades of research.
  arXiv  Detail & Related papers  (2025-11-23T03:17:26Z)
• DispatchMAS: Fusing taxonomy and artificial intelligence agents for emergency medical services [49.70819009392778]
  Large Language Models (LLMs) and Multi-Agent Systems (MAS) offer opportunities to augment dispatchers.<n>This study aimed to develop and evaluate a taxonomy-grounded, multi-agent system for simulating realistic scenarios.
  arXiv  Detail & Related papers  (2025-10-24T08:01:21Z)
• LLM-Based Agents for Competitive Landscape Mapping in Drug Asset Due Diligence [0.231749519765552]
  We describe and benchmark a competitor-discovery component used within an agentic AI system for fast drug asset due diligence.<n>A competitor-discovery AI agent, given an indication, retrieves all drugs comprising the competitive landscape of that indication.<n>Our competitor-discovery agent achieves 83% recall, exceeding OpenAI Deep Research and Perplexity Labs.
  arXiv  Detail & Related papers  (2025-08-22T17:50:00Z)
• DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discovery [54.79763887844838]
  Large language models (LLMs) integrated with autonomous agents hold significant potential for advancing scientific discovery through automated reasoning and task execution.<n>We introduce DrugPilot, a LLM-based agent system with a parameterized reasoning architecture designed for end-to-end scientific in drug discovery.<n>DrugPilot significantly outperforms state-of-the-art agents such as ReAct and LoT, achieving task completion rates of 98.0%, 93.5%, and 64.0% for simple, multi-tool, and multi-turn scenarios, respectively.
  arXiv  Detail & Related papers  (2025-05-20T05:18:15Z)
• MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models [82.30696225661615]
  We introduce MedHallu, the first benchmark specifically designed for medical hallucination detection.<n>We show that state-of-the-art LLMs, including GPT-4o, Llama-3.1, and medically fine-tuned UltraMedical, struggle with this binary hallucination detection task.<n>Using bidirectional entailment clustering, we show that harder-to-detect hallucinations are semantically closer to ground truth.
  arXiv  Detail & Related papers  (2025-02-20T06:33:23Z)
• Neural Bandits for Data Mining: Searching for Dangerous Polypharmacy [63.135687276599114]
  Some polypharmacies, deemed inappropriate, may be associated with adverse health outcomes such as death or hospitalization. We propose the OptimNeuralTS strategy to efficiently mine claims datasets and build a predictive model of the association between drug combinations and health outcomes. Our method can detect up to 72% of PIPs while maintaining an average precision score of 99% using 30 000 time steps.
  arXiv  Detail & Related papers  (2022-12-10T03:43:23Z)
• Knowledge-Driven New Drug Recommendation [88.35607943144261]
  We develop a drug-dependent multi-phenotype few-shot learner to bridge the gap between existing and new drugs. EDGE eliminates the false-negative supervision signal using an external drug-disease knowledge base. Results show that EDGE achieves 7.3% improvement on the ROC-AUC score over the best baseline.
  arXiv  Detail & Related papers  (2022-10-11T16:07:52Z)
• An Integrated System of Drug Matching and Abnormal Approval Number Correction [0.0]
  This paper creates an integrated system for matching drug products from two data sources. Our integrated system achieves 98.3% drug matching accuracy, with 99.2% precision and 97.5% recall.
  arXiv  Detail & Related papers  (2022-07-01T11:19:50Z)
• Principal-Agent Hypothesis Testing [54.154244569974864]
  We consider the relationship between a regulator (the principal) and an experimenter (the agent) such as a pharmaceutical company. The efficacy of the drug is not known to the regulator, so the pharmaceutical company must run a costly trial to prove efficacy to the regulator. We show how to design protocols that are robust to an agent's strategic actions, and derive the optimal protocol in the presence of strategic entrants.
  arXiv  Detail & Related papers  (2022-05-13T17:59:23Z)
• Extracting Adverse Drug Events from Clinical Notes [1.6244541005112747]
  Adverse drug events (ADEs) are unexpected incidents caused by the administration of a drug or medication. This paper explores the relationship between a drug and its associated attributes using relation extraction techniques.
  arXiv  Detail & Related papers  (2021-04-21T23:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.