Robust or Suggestible? Exploring Non-Clinical Induction in LLM Drug-Safety Decisions

• URL: http://arxiv.org/abs/2510.13931v1
• Date: Wed, 15 Oct 2025 14:11:11 GMT
• Title: Robust or Suggestible? Exploring Non-Clinical Induction in LLM Drug-Safety Decisions
• Authors: Siying Liu, Shisheng Zhang, Indu Bala,
• Abstract summary: Large language models (LLMs) are increasingly applied in biomedical domains, yet their reliability in drug-safety prediction remains underexplored.<n>We assess two state-of-the-art models, ChatGPT-4o and Bio-Medical-Llama-3.8B, across diverse personas defined by education, marital status, employment, insurance, language, housing stability, and religion.
• Score: 5.347177139463896
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly applied in biomedical domains, yet their reliability in drug-safety prediction remains underexplored. In this work, we investigate whether LLMs incorporate socio-demographic information into adverse event (AE) predictions, despite such attributes being clinically irrelevant. Using structured data from the United States Food and Drug Administration Adverse Event Reporting System (FAERS) and a persona-based evaluation framework, we assess two state-of-the-art models, ChatGPT-4o and Bio-Medical-Llama-3.8B, across diverse personas defined by education, marital status, employment, insurance, language, housing stability, and religion. We further evaluate performance across three user roles (general practitioner, specialist, patient) to reflect real-world deployment scenarios where commercial systems often differentiate access by user type. Our results reveal systematic disparities in AE prediction accuracy. Disadvantaged groups (e.g., low education, unstable housing) were frequently assigned higher predicted AE likelihoods than more privileged groups (e.g., postgraduate-educated, privately insured). Beyond outcome disparities, we identify two distinct modes of bias: explicit bias, where incorrect predictions directly reference persona attributes in reasoning traces, and implicit bias, where predictions are inconsistent, yet personas are not explicitly mentioned. These findings expose critical risks in applying LLMs to pharmacovigilance and highlight the urgent need for fairness-aware evaluation protocols and mitigation strategies before clinical deployment.

Related papers

• Uncovering Latent Bias in LLM-Based Emergency Department Triage Through Proxy Variables [2.9269181918140643]
  We investigate bias in large language models (LLMs)-based medical AI systems applied to emergency department (ED) triage.<n>Our results reveal discriminatory behavior mediated through proxy variables in ED triage scenarios.<n>These findings indicate that AI systems is still imperfectly trained on noisy, sometimes non-causal signals that do not reliably reflect true patient acuity.
  arXiv  Detail & Related papers  (2026-01-13T07:49:41Z)
• Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models [52.2001050216955]
  Existing methods aim to enhance the performance of Medical Vision Language Model (MedVLM) by adjusting model structure, fine-tuning with high-quality data, or through preference fine-tuning.<n>We propose an expert-in-the-loop framework named Expert-Controlled-Free Guidance (Expert-CFG) to align MedVLM with clinical expertise without additional training.
  arXiv  Detail & Related papers  (2025-07-12T09:03:30Z)
• Predictive Representativity: Uncovering Racial Bias in AI-based Skin Cancer Detection [0.0]
  This paper introduces the concept of Predictive Representativity (PR)<n>PR shifts the focus from the composition of the data set to outcomes-level equity.<n>Our analysis reveals substantial performance disparities by skin phototype.
  arXiv  Detail & Related papers  (2025-07-10T22:21:06Z)
• Affective-ROPTester: Capability and Bias Analysis of LLMs in Predicting Retinopathy of Prematurity [34.80765908439636]
  Large language models' capacity to predict retinopathy of prematurity (ROP) risk remains largely unexplored.<n>We introduce a novel Chinese benchmark dataset, termed CROP, comprising 993 admission records annotated with low, medium, and high-risk labels.<n>We propose Affective-ROPTester, an automated evaluation framework incorporating three prompting strategies.
  arXiv  Detail & Related papers  (2025-07-08T09:36:14Z)
• Bias in Large Language Models Across Clinical Applications: A Systematic Review [0.0]
  Large language models (LLMs) are rapidly being integrated into healthcare, promising to enhance various clinical tasks.<n>This systematic review investigates the prevalence, sources, manifestations, and clinical implications of bias in LLMs.
  arXiv  Detail & Related papers  (2025-04-03T13:32:08Z)
• Uncertainty-aware abstention in medical diagnosis based on medical texts [87.88110503208016]
  This study addresses the critical issue of reliability for AI-assisted medical diagnosis.<n>We focus on the selection prediction approach that allows the diagnosis system to abstain from providing the decision if it is not confident in the diagnosis.<n>We introduce HUQ-2, a new state-of-the-art method for enhancing reliability in selective prediction tasks.
  arXiv  Detail & Related papers  (2025-02-25T10:15:21Z)
• Comparing Targeting Strategies for Maximizing Social Welfare with Limited Resources [20.99198458867724]
  Policymakers rarely have access to data from a randomized controlled trial (RCT) that would enable accurate estimates of which individuals would benefit more from the intervention.<n> Practitioners instead commonly use a technique termed risk-based targeting" where the model is just used to predict each individual's status quo outcome.<n>There is currently almost no empirical evidence to inform which choices lead to the most effective machine learning-informed targeting strategies.
  arXiv  Detail & Related papers  (2024-11-11T22:36:50Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Clinical Risk Prediction Using Language Models: Benefits And Considerations [23.781690889237794]
  This study focuses on using structured descriptions within vocabularies to make predictions exclusively based on that information. We find that employing LMs to represent structured EHRs leads to improved or at least comparable performance in diverse risk prediction tasks.
  arXiv  Detail & Related papers  (2023-11-29T04:32:19Z)
• Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes [50.8044927215346]
  We consider a comorbidity risk prediction scenario and focus on contexts regarding the patients clinical state. We employ several state-of-the-art LLMs to present contexts around risk prediction model inferences and evaluate their acceptability. Our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case.
  arXiv  Detail & Related papers  (2023-02-11T18:07:11Z)
• Statistical discrimination in learning agents [64.78141757063142]
  Statistical discrimination emerges in agent policies as a function of both the bias in the training population and of agent architecture. We show that less discrimination emerges with agents that use recurrent neural networks, and when their training environment has less bias.
  arXiv  Detail & Related papers  (2021-10-21T18:28:57Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.