TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology

• URL: http://arxiv.org/abs/2606.19245v2
• Date: Thu, 18 Jun 2026 02:34:51 GMT
• Title: TxBench-PP: Analyzing AI Agent Performance on Small-Molecule Preclinical Pharmacology
• Abstract summary: Artificial intelligence (AI) agents promise to accelerate drug discovery by compressing interpretation and decision-making loops.<n>We introduce TherapeuticsBench Preclinical Pharmacology (TxBench-PP), a verifiable benchmark for small-molecule preclinical pharmacology.<n>TxBench-PP tests whether agents can recover accurate conclusions from real-world assay data rather than memorized facts from literature.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial intelligence (AI) agents promise to accelerate drug discovery by compressing interpretation and decision-making loops, but practical deployment requires trusted evaluation on realistic program decisions. We introduce TherapeuticsBench Preclinical Pharmacology (TxBench-PP), a verifiable benchmark for small-molecule preclinical pharmacology and the first focused slice of a broader TherapeuticsBench effort across drug-discovery stages and therapeutic modalities. TxBench-PP tests whether agents can recover accurate conclusions from real-world assay data rather than memorized facts from literature. The benchmark contains 100 evaluations indexed by program stage, assay type, and task structure, spanning mechanism-of-action (MoA) and pharmacodynamic (PD) reasoning, compound-target engagement, causal target validation, developability and safety, and translational efficacy. Agents receive realistic workflow snapshots, inspect files in a coding environment, and return structured answers graded deterministically. Across 16 model-harness configurations, comprising 11 models and 4,800 trajectories, no system reliably recovered preclinical pharmacology decisions. The strongest configuration, Claude Opus 4.8 / Pi, passed 59.3\% of endpoint attempts (178/300; 95\% CI, 51.1-67.6), followed by GPT-5.5 / Pi at 55.3\% (166/300; 47.0-63.6).

Related papers

• Evaluating Large Language Models in Dynamic Clinical Decision-Making with Standardized Patient Cases [71.12461204050985]
  MedSP1000 is an SP-derived interactive benchmark for clinical-agent evaluation.<n>It converts peer-reviewed SP teaching cases into executable scenarios with defined SP case scripts, clinical environment contexts, and human-validated structured rubrics.<n>Applying MedSP1000 to a range of general-purpose and medically specialized LLMs, we find that performance on static benchmarks does not reliably translate to such educational scenarios.
  arXiv  Detail & Related papers  (2026-06-03T17:17:16Z)
• Eligibility-Aware Evidence Synthesis: An Agentic Framework for Clinical Trial Meta-Analysis [49.83108591873481]
  EligMeta is an agentic framework that integrates automated trial discovery with eligibility-aware meta-analysis.<n>The framework structures eligibility criteria and computes similarity-based study weights reflecting population alignment between target trials.
  arXiv  Detail & Related papers  (2026-04-03T03:18:50Z)
• Agentic Cognitive Profiling: Realigning Automated Alzheimer's Disease Detection with Clinical Construct Validity [66.94391219005291]
  This paper proposes Agentic Cognitive Profiling (ACP), an agentic framework that realigns automated screening with clinical protocol logic.<n>Central to our design is decoupling semantic understanding from measurement by delegating all quantification to deterministic function calling.<n>ACP achieves 90.5% score match rate in task examination and 85.3% accuracy in AD prediction, surpassing popular baselines.
  arXiv  Detail & Related papers  (2026-03-18T06:15:35Z)
• MedAI: Evaluating TxAgent's Therapeutic Agentic Reasoning in the NeurIPS CURE-Bench Competition [6.191248426050678]
  Therapeutic decision-making in clinical medicine requires robust, multi-step reasoning grounded in reliable biomedical knowledge.<n>Agentic AI methods, exemplified by TxAgent, address these challenges through iterative retrieval-augmented generation (RAG)<n>This work presents insights derived from our participation in the CURE-Bench NeurIPS 2025 Challenge, which benchmarks therapeutic-reasoning systems.
  arXiv  Detail & Related papers  (2025-12-12T16:01:48Z)
• Retrieval-Augmented Generation Assistant for Anatomical Pathology Laboratories [2.125786478579275]
  This study proposes and evaluates a Retrieval-Augmented Generation (RAG) assistant tailored to Anatomical Pathology (AP) laboratories.<n>We curated a novel corpus of 99 AP protocols from a Portuguese healthcare institution and constructed 323 question-answer pairs.<n>Retrieval-Augmented Generation assistant provides technicians with context-grounded answers to protocol-related queries.
  arXiv  Detail & Related papers  (2025-12-08T22:44:57Z)
• DispatchMAS: Fusing taxonomy and artificial intelligence agents for emergency medical services [49.70819009392778]
  Large Language Models (LLMs) and Multi-Agent Systems (MAS) offer opportunities to augment dispatchers.<n>This study aimed to develop and evaluate a taxonomy-grounded, multi-agent system for simulating realistic scenarios.
  arXiv  Detail & Related papers  (2025-10-24T08:01:21Z)
• MedPAO: A Protocol-Driven Agent for Structuring Medical Reports [0.13029689752120577]
  We introduce MedPAO, a novel agentic framework that ensures accuracy and verifiable reasoning.<n> MedPAO decomposes the report structuring task into a transparent process managed by a Plan-Act-Observe (PAO) loop and specialized tools.<n>The efficacy of our approach is demonstrated through rigorous evaluation: MedPAO achieves an F1-score of 0.96 on the critical sub-task of concept categorization.
  arXiv  Detail & Related papers  (2025-10-06T09:32:23Z)
• From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations [45.414878840652115]
  Large language models (LLMs) have demonstrated promising performance on medical benchmarks.<n>However, their ability to perform medical calculations remains underexplored and poorly evaluated.<n>In this work, we revisit medical calculation evaluation with a stronger focus on clinical trustworthiness.
  arXiv  Detail & Related papers  (2025-09-20T09:10:26Z)
• DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discovery [54.79763887844838]
  Large language models (LLMs) integrated with autonomous agents hold significant potential for advancing scientific discovery through automated reasoning and task execution.<n>We introduce DrugPilot, a LLM-based agent system with a parameterized reasoning architecture designed for end-to-end scientific in drug discovery.<n>DrugPilot significantly outperforms state-of-the-art agents such as ReAct and LoT, achieving task completion rates of 98.0%, 93.5%, and 64.0% for simple, multi-tool, and multi-turn scenarios, respectively.
  arXiv  Detail & Related papers  (2025-05-20T05:18:15Z)
• TxAgent: An AI Agent for Therapeutic Reasoning Across a Universe of Tools [22.322166889507184]
  TxAgent is an AI agent that analyzes drug interactions, contraindications, and patient-specific treatment strategies.<n>The ToolUniverse consolidates 211 tools from trusted sources, including all US FDA-approved drugs since 1939.<n>It achieves 92.1% accuracy in open-ended drug reasoning tasks, surpassing GPT-4o and outperforming DeepSeek-R1 (671B) in structured multi-step reasoning.
  arXiv  Detail & Related papers  (2025-03-14T00:28:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.