MedUPS: Towards Diagnostic Assistance in Uncommon Medical Cases with Large Language Models

• URL: http://arxiv.org/abs/2608.01012v1
• Date: Sun, 02 Aug 2026 05:27:21 GMT
• Title: MedUPS: Towards Diagnostic Assistance in Uncommon Medical Cases with Large Language Models
• Abstract summary: MedUPSQA is a dataset of 21,874 mid-stream clinical decision points built from 5,535 real case reports.<n>MedUPS supervises models on intermediate decisions as they unfold along a patient's trajectory.
• Score: 9.0377422390685
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Uncommon and off-guideline cases are difficult for clinical decision support, because physicians must make a series of management decisions under diagnostic uncertainty and rarely see the full case at once. Most large language model (LLM) benchmarks for medicine score only the final diagnosis, yet much of clinical care turns on the next appropriate action: the next test to order, the imaging study to obtain, the specialist to involve, or the differential to pursue. We introduce MedUPSQA, a dataset of 21,874 mid-stream clinical decision points built from 5,535 real case reports, and MedUPS, an alignment framework that supervises models on these intermediate decisions as they unfold along a patient's trajectory. We segment free-text case presentations into chronologically ordered, accumulating clinical chunks and align models to predict the next step with reinforcement learning (GRPO), using an external LLM-as-a-Judge reward. This objective mirrors how clinicians actually meet patients, reasoning forward from accumulating evidence toward the next decision, rather than committing to a final label. Across three backbones, mid-stream alignment raises next-step accuracy from 55.2 to 66.7 for Qwen3.6-27B, from 47.2 to 57.8 for Qwen3.5-9B, and from 37.8 to 44.4 for HuatuoGPT-3-8B, with 95% CI. In several model scales we test the objective improves accuracy more than scale, with smaller models surpassing larger, frontier models we evaluate. We further train supervised fine-tuning (SFT) baselines on the mid-stream task, SFT improves all backbones above base, indicating the target framwork carries signal independently of the optimizer. We release the dataset, code, and aligned checkpoints.

Related papers

• ChronoMedicalWorld: A Medical World Model for Learning Patient Trajectories from Longitudinal Care Data [1.995753801570114]
  We propose an action-conditioned latent world-model framework for learning patient trajectories from longitudinal care data.<n>We instantiate CMWM for annual estimated glomerular filtration rate (eGFR) trajectory forecasting in chronic kidney disease (CKD)<n>The framework is not CKD-specific its architecture, loss design, and training protocol apply to any chronic condition that can be cast as periodic clinical state interleaved with structured and conversational interventions.
  arXiv  Detail & Related papers  (2026-05-21T03:50:17Z)
• MedAction: Towards Active Multi-turn Clinical Diagnostic LLMs [11.709211007913803]
  Existing medical training data teaches models to reason from complete information but not to act under evolving, partial evidence.<n>We introduce MedAction, a tree-structured distillation pipeline that synthesizes diverse and high-quality multi-turn diagnostic trajectories.<n>Using this pipeline, we construct MedAction-32K, a dataset of 32,681 trajectories from 2,896 PMC cases.
  arXiv  Detail & Related papers  (2026-05-08T06:15:24Z)
• MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning [7.000170880015254]
  We propose MedPRMBench, the first process-level reward model benchmark for the medical domain.<n>It is built through a three-phase pipeline based on Clinical Reasoning Blueprints (CRBs)<n>The benchmark comprises 6,500 questions with 13,000 reasoning chains and 113,910 step-level labels, plus 6,879 questions for training.
  arXiv  Detail & Related papers  (2026-04-19T06:44:07Z)
• Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction [72.89352076103889]
  Large language models (LLMs) achieve high accuracy in medical diagnosis when all clinical information is provided in a single turn.<n>We introduce MINT, a high-fidelity, multi-turn medical diagnosis benchmark comprising 1,035 cases.<n>We uncover three persistent behavioral patterns that significantly impact diagnostic decisions.
  arXiv  Detail & Related papers  (2026-04-06T00:23:10Z)
• Clinician input steers frontier AI models toward both accurate and harmful decisions [10.599240857217811]
  We evaluated 21 large language models (LLMs) across 8 frontier models on differential diagnosis generation and next step recommendations.<n>Expert context significantly improved correct final diagnosis inclusion across all 21 models.<n>In GPT-4o experiments, explicit clinician uncertainty signals improved diagnostic performance after adversarial context.
  arXiv  Detail & Related papers  (2026-03-14T23:47:53Z)
• Evolving Diagnostic Agents in a Virtual Clinical Environment [75.59389103511559]
  We present a framework for training large language models (LLMs) as diagnostic agents with reinforcement learning.<n>Our method acquires diagnostic strategies through interactive exploration and outcome-based feedback.<n>DiagAgent significantly outperforms 10 state-of-the-art LLMs, including DeepSeek-v3 and GPT-4o.
  arXiv  Detail & Related papers  (2025-10-28T17:19:47Z)
• EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models [82.43729208063468]
  Recent benchmarks for medical Large Vision-Language Models (LVLMs) emphasize leaderboard accuracy, overlooking reliability and safety.<n>We study sycophancy -- models' tendency to uncritically echo user-provided information.<n>We introduce EchoBench, a benchmark to systematically evaluate sycophancy in medical LVLMs.
  arXiv  Detail & Related papers  (2025-09-24T14:09:55Z)
• Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models [57.73472878679636]
  We introduce Med-RewardBench, the first benchmark specifically designed to evaluate medical reward models and judges.<n>Med-RewardBench features a multimodal dataset spanning 13 organ systems and 8 clinical departments, with 1,026 expert-annotated cases.<n>A rigorous three-step process ensures high-quality evaluation data across six clinically critical dimensions.
  arXiv  Detail & Related papers  (2025-08-29T08:58:39Z)
• A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment [46.776978552161395]
  Small language models (SLMs) offer a cost-effective alternative to large language models such as GPT-4.<n>SLMs offer a cost-effective alternative, but their limited capacity requires biomedical domain adaptation.<n>We propose a novel framework for adapting SLMs into high-performing clinical models.
  arXiv  Detail & Related papers  (2025-05-15T21:40:21Z)
• Towards Evaluating and Building Versatile Large Language Models for Medicine [57.49547766838095]
  We present MedS-Bench, a benchmark designed to evaluate the performance of large language models (LLMs) in clinical contexts. MedS-Bench spans 11 high-level clinical tasks, including clinical report summarization, treatment recommendations, diagnosis, named entity recognition, and medical concept explanation. MedS-Ins comprises 58 medically oriented language corpora, totaling 13.5 million samples across 122 tasks.
  arXiv  Detail & Related papers  (2024-08-22T17:01:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.