An AI agent for treatment reasoning over a biomedical tool universe
Abstract Overview
This paper introduces ATHENA-R1, an AI agent for treatment reasoning that iteratively gathers and integrates evidence from a library of 212 biomedical tools. Rather than relying solely on parametric knowledge, the system identifies missing information, retrieves evidence such as FDA label content, and uses it to update its reasoning step by step. The training pipeline combines self-generated reasoning traces with reinforcement learning using scientific feedback, eliminating the need for human-annotated reasoning demonstrations. The system is comprehensively evaluated on benchmark drug-reasoning tasks, blinded expert reviews for rare diseases, physician-rated hospital cases, and population-scale electronic health record validations.
Novelty
The primary novelty lies in formulating treatment reasoning as a learnable, multi-step evidence-seeking process over a large biomedical tool universe rather than direct answer generation from a language model. Additionally, it introduces a two-stage training scheme utilizing multi-agent self-learning and reinforcement learning with scientific feedback to build reasoning traces and tool-use behaviors without human annotations.
Results
On benchmark evaluations, the agent achieved 94.7% accuracy on DrugPC and 82.9% on TreatmentPC, outperforming models such as GPT-5 and DeepSeek-R1. In blinded expert evaluations, its responses were consistently preferred across clinical criteria, and it successfully reasoned through complex hospitalized-patient cases. Furthermore, retrospective EHR analyses of 5.4 million patients validated its generated adverse-event hypotheses, with predicted associations reaching adjusted odds ratios between 1.48 and 1.84.
Key Points
- ATHENA-R1 performs multi-step treatment reasoning by selecting and executing tools from a 212-tool biomedical library in real time to produce evidence-grounded reasoning traces.
- The system is trained using a two-level scheme that combines a self-generated instruction dataset with reinforcement learning based on multiaxial scientific feedback.
- Empirical evaluation demonstrates state-of-the-art performance across controlled benchmarks, blinded expert assessments, physician reviews of complex cases, and population-scale EHR hypothesis validation.