The Value Sensitivity Gap: How Clinical Large Language Models Respond to Patient Preference Statements in Shared Decision-Making

• URL: http://arxiv.org/abs/2603.00076v1
• Date: Thu, 12 Feb 2026 21:42:21 GMT
• Title: The Value Sensitivity Gap: How Clinical Large Language Models Respond to Patient Preference Statements in Shared Decision-Making
• Abstract summary: Large language models (LLMs) are entering clinical as decision support tools, yet how they respond to explicit patient value statements remains unmeasured.<n>We conducted a factorial experiment using clinical vignettes derived from 98,759 de-identified Medicaid encounter notes.<n>All models acknowledged patient values in 100% of non-control trials, yet actual recommendation shifting remained modest.
• Score: 0.6768558752130311
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are entering clinical workflows as decision support tools, yet how they respond to explicit patient value statements -- the core content of shared decision-making -- remains unmeasured. We conducted a factorial experiment using clinical vignettes derived from 98,759 de-identified Medicaid encounter notes. We tested four LLM families (GPT-5.2, Claude 4.5 Sonnet, Gemini 3 Pro, and DeepSeek-R1) across 13 value conditions in two clinical domains, yielding 104 trials. Default value orientations differed across model families (aggressiveness range 2.0 to 3.5 on a 1-to-5 scale). Value sensitivity indices ranged from 0.13 to 0.27, and directional concordance with patient-stated preferences ranged from 0.625 to 1.0. All models acknowledged patient values in 100% of non-control trials, yet actual recommendation shifting remained modest. Decision-matrix and VIM self-report mitigations each improved directional concordance by 0.125 in a 78-trial Phase 2 evaluation. These findings provide empirical data for populating value disclosure labels proposed by clinical AI governance frameworks.

Related papers

• What EEG Foundation Models Encode: Dataset Identity and a Negative-Control Suite for Clinical Benchmarks [0.0]
  We benchmark LaBraM, EEGMamba, CBraMod, REVE, LEAD, BENDR, and BIOT on five clinical tasks across four datasets.<n>We challenge apparent gains using stronger classical comparators, label permutation, scrambled-label fine-tuning, and random-initialisation controls.<n>Conclusions change materially after montage matching, patient-overlap checks, stronger comparators, and representation controls.
  arXiv  Detail & Related papers  (2026-07-27T14:59:42Z)
• Clinician-Level Agreement Without Clinical Caution: LLM Evaluator Limits in Medical AI Benchmarking [27.829988205900055]
  We introduce MedQADE, the first standardised open-response clinical benchmark for German.<n>The top-performing evaluator model, Gemini 3 Flash, reached alignment consistent with the physician ceiling.<n>Despite this statistical alignment, automated evaluators exhibited near-absent clinical metacognition.
  arXiv  Detail & Related papers  (2026-07-01T15:55:31Z)
• Beyond Scalar Scores: Exploring LLM-based Metrics for Clinical Significance Evaluation in Radiology Reports [49.5225801722164]
  Existing metrics obscure this requirement by reducing report quality to a medically ungrounded scalar.<n>We study this boundary using ReEvalMed benchmark as testbed and evaluate metric-level clinical significance.
  arXiv  Detail & Related papers  (2026-06-17T08:10:30Z)
• A Decade-Scale Benchmark Evaluating LLMs' Clinical Practice Guidelines Detection and Adherence in Multi-turn Conversations [60.2076951536797]
  Large Language Models (LLMs) are increasingly deployed in healthcare scenarios.<n>It is unclear to which extend LLMs could identify and adhere to clinical guidelines during conversations.<n>CPGBench is an automated framework benchmarking the clinical guideline detection and adherence capabilities of LLMs.
  arXiv  Detail & Related papers  (2026-03-26T09:00:55Z)
• Cerebra: A Multidisciplinary AI Board for Multimodal Dementia Characterization and Risk Assessment [56.62016795093786]
  Cerebra is an interactive multi-agent AI team that coordinates specialized agents for EHR, clinical notes, and medical imaging analysis.<n>It supports privacy-preserving deployment by operating on structured representations and remains robust when modalities are incomplete.<n>Cerebra significantly improved expert performance, increasing accuracy by 17.5 percentage points in prospective dementia risk estimation.
  arXiv  Detail & Related papers  (2026-03-23T05:46:45Z)
• Clinician input steers frontier AI models toward both accurate and harmful decisions [10.599240857217811]
  We evaluated 21 large language models (LLMs) across 8 frontier models on differential diagnosis generation and next step recommendations.<n>Expert context significantly improved correct final diagnosis inclusion across all 21 models.<n>In GPT-4o experiments, explicit clinician uncertainty signals improved diagnostic performance after adversarial context.
  arXiv  Detail & Related papers  (2026-03-14T23:47:53Z)
• MedArena: Comparing LLMs for Medicine-in-the-Wild Clinician Preferences [50.71326426975699]
  We present MedArena, an interactive evaluation platform for medical large language models (LLMs)<n>Given a clinician-provided query, MedArena presents responses from two randomly selected models and asks the user to select the preferred response.<n>Out of 1571 preferences collected across 12 LLMs up to November 1, 2025, Gemini 2.0 Flash Thinking, Gemini 2.5 Pro, and GPT-4o were the top three models by Bradley-Terry rating.
  arXiv  Detail & Related papers  (2026-03-13T22:30:26Z)
• PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology [48.732366302949515]
  Large language models (LLMs) have achieved expert-level performance on standardized examinations, yet multiple-choice accuracy poorly reflects real-world clinical utility and safety.<n>We developed a human-in-the-loop pipeline to create expert rubrics for de-identified patient questions.<n>We evaluated 22 proprietary and open-source LLMs using an LLM-as-a-judge framework, measuring clinical completeness, factual accuracy, and web-search integration.
  arXiv  Detail & Related papers  (2026-03-02T00:50:39Z)
• Conformal Prediction for Risk-Controlled Medical Entity Extraction Across Clinical Domains [2.744879848686032]
  We present a conformal prediction framework that provides finite-sample coverage guarantees for medical entity extraction.<n>First, we extract structured entities from 1,000 FDA drug labels across eight sections using GPT-4.1, verified via FactScore-based atomic statement evaluation.<n>Second, we extract radiological entities from MIMIC-CXR reports using the RadGraph schema with GPT-4.1 and Llama-4-Maverick, evaluated against physician annotations.
  arXiv  Detail & Related papers  (2026-03-01T05:12:25Z)
• LiveClin: A Live Clinical Benchmark without Leakage [50.45415584327275]
  LiveClin is a live benchmark designed for approximating real-world clinical practice.<n>We transform authentic patient cases into complex, multimodal evaluation scenarios that span the entire clinical pathway.<n>Our evaluation of 26 models on LiveClin reveals the profound difficulty of these real-world scenarios, with the top-performing model achieving a Case Accuracy of just 35.7%.
  arXiv  Detail & Related papers  (2026-02-18T03:59:46Z)
• Explainable Admission-Level Predictive Modeling for Prolonged Hospital Stay in Elderly Populations: Challenges in Low- and Middle-Income Countries [65.4286079244589]
  Prolonged length of stay (pLoS) is a significant factor associated with the risk of adverse in-hospital events.<n>We develop and explain a predictive model for pLos using admission-level patient and hospital administrative data.
  arXiv  Detail & Related papers  (2026-01-07T23:35:24Z)
• DeVisE: Behavioral Testing of Medical Large Language Models [14.832083455439749]
  DeVisE is a behavioral testing framework for probing fine-grained clinical understanding.<n>We construct a dataset of ICU discharge notes from MIMIC-IV.<n>We evaluate five LLMs spanning general-purpose and medically fine-tuned variants.
  arXiv  Detail & Related papers  (2025-06-18T10:42:22Z)
• MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks [47.486705282473984]
  Large language models (LLMs) achieve near-perfect scores on medical exams.<n>These evaluations inadequately reflect complexity and diversity of real-world clinical practice.<n>We introduce MedHELM, an evaluation framework for assessing LLM performance for medical tasks.
  arXiv  Detail & Related papers  (2025-05-26T22:55:49Z)
• Can Reasoning LLMs Enhance Clinical Document Classification? [7.026393789313748]
  Large Language Models (LLMs) offer promising improvements in accuracy and efficiency for this task.<n>This study evaluates the performance and consistency of eight LLMs; four reasoning (Qwen QWQ, Deepseek Reasoner, GPT o3 Mini, Gemini 2.0 Flash Thinking) and four non-reasoning (Llama 3.3, GPT 4o Mini, Gemini 2.0 Flash, Deepseek Chat)<n>Results showed that reasoning models outperformed non-reasoning models in accuracy (71% vs 68%) and F1 score (67% vs 60%)
  arXiv  Detail & Related papers  (2025-04-10T18:00:27Z)
• Multi-modal AI for comprehensive breast cancer prognostication [18.691704371847855]
  We developed a novel artificial intelligence (AI)-based approach that integrates digital pathology with clinical data.<n>The test was developed and evaluated using data from a total of 8,161 female breast cancer patients across 15 cohorts.<n>Our results suggest that our AI test improves upon the accuracy of existing prognostic tests, while being applicable to a wider range of patients.
  arXiv  Detail & Related papers  (2024-10-28T17:54:29Z)
• Classification supporting COVID-19 diagnostics based on patient survey data [82.41449972618423]
  logistic regression and XGBoost classifiers, that allow for effective screening of patients for COVID-19 were generated. The obtained classification models provided the basis for the DECODE service (decode.polsl.pl), which can serve as support in screening patients with COVID-19 disease. This data set consists of more than 3,000 examples is based on questionnaires collected at a hospital in Poland.
  arXiv  Detail & Related papers  (2020-11-24T17:44:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.