The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions

• URL: http://arxiv.org/abs/2608.05583v1
• Date: Thu, 06 Aug 2026 04:12:05 GMT
• Title: The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions
• Abstract summary: Large language models (LLMs) enter high-stakes domains such as healthcare.<n>LLMs agree with humans that patients bear responsibility for health-harming behaviors, yet overwhelmingly refuse to let that judgment influence how they allocate scarce resources.<n>LLMs apply a systematically different moral framework than humans when responsibility and resource scarcity intersect.
• Score: 20.192422641852534
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) enter high-stakes domains such as healthcare, understanding their moral reasoning becomes essential. Decisions about scarce medical resources often hinge on judgments of responsibility, particularly when patients' own actions contribute to illness. We investigate how LLMs reason about responsibility and its consequences, tracing their judgments across successive levels, from the behavior, to the resulting illness, to the denial of care. We evaluate a wide range of LLMs, spanning different model families and capability levels, on various clinical vignettes adapted from prior studies. Our results identify a judgment-consequence gap: LLMs largely agree with humans that patients bear responsibility for health-harming behaviors, yet overwhelmingly refuse to let that judgment influence how they allocate scarce resources. Specifically, LLMs default to random allocation, whereas humans consistently favor the less-culpable patient. Compared to humans, LLMs also place greater emphasis on access to information, reducing responsibility judgments when health-risk knowledge is unavailable. These findings reveal that LLMs apply a systematically different moral framework than humans when responsibility and resource scarcity intersect, surprisingly often amplifying normative disagreement with humans as reasoning capability increases.

Related papers

• How do LLMs Evaluate Perceived Moral Agency? Investigating Moral Decision-Making in Human-Artificial Agents Interactions [66.41429744325532]
  Humans possess moral agency, the capacity to make ethically guided decisions and bear responsibility for their consequences.<n>As artificial agents (AAs) such as robots, drones, and disembodied AI systems become embedded in smart city environments, the question of whether and how moral agency is attributed to them takes on new urgency.<n>This paper presents, to the best of our knowledge, the first empirical study comparing how humans and LLMs evaluate perceived moral agency (PMA) across human and autonomous artificial agents varying in embodiment.
  arXiv  Detail & Related papers  (2026-09-04T11:56:59Z)
• Rare Diseases, Common Dilemmas: LLMs Prioritize Equal Resource Distribution over Patient Benefit in Decision-Making [6.669841955172372]
  Clinical decision-making often involves prioritizing ethical values, such as beneficence, non-maleficence, respecting a patient's autonomy, and justice.<n>Recent work has begun to assess how large language models (LLMs) make such subjective, value-laden clinical judgments.<n>We present a benchmark of 208 clinically grounded rare disease vignettes, each of which presents genuine, high-stakes conflicts.
  arXiv  Detail & Related papers  (2026-08-25T23:54:22Z)
• Characterizing Rhetorical Misalignment in Decision-Making with Language Models [22.79503625028684]
  Large language models (LLMs) are increasingly integrated into high-stakes human-AI decision-making.<n>We study rhetorical misalignment, a failure mode where an LLM uses rhetorically inappropriate forms of presentation for a given decision context.<n>We observe that LLMs induce an average 2.81% rate of harmful decision flips across different models.
  arXiv  Detail & Related papers  (2026-07-24T02:36:00Z)
• Can Induced Emotion Bias LLM Behaviors in Sequential Decision Making? [64.5408523407693]
  Large Language Models (LLMs) are increasingly deployed as autonomous agents in high-stakes domains.<n>We investigate whether induced emotion can influence their sequential decision-making using the Iowa Gambling Task.<n>We find that, different from humans, induced emotion does not significantly bias the decision dynamics of LLM agents on average.
  arXiv  Detail & Related papers  (2026-07-14T11:09:51Z)
• Verbalizing LLMs' assumptions to explain and control sycophancy [62.927670321859495]
  LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment.<n>We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs.<n>Our work contributes a new understanding of assumptions as a mechanism for sycophancy.
  arXiv  Detail & Related papers  (2026-04-03T14:15:43Z)
• LLMs in Cybersecurity: Friend or Foe in the Human Decision Loop? [0.15293427903448023]
  Large Language Models (LLMs) are transforming human decision-making by acting as cognitive collaborators.<n>This paper investigates how LLMs shape human judgment in security-critical contexts.
  arXiv  Detail & Related papers  (2025-09-08T12:06:06Z)
• Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Values [36.47201247038004]
  We show that Large Language Models (LLMs) exhibit stark deviations from human values in prioritizing various attributes.<n>We show that low-rank supervised fine-tuning with few samples is often effective in improving both decision consistency and calibrating indecision modeling.
  arXiv  Detail & Related papers  (2025-05-30T01:23:11Z)
• Fact or Guesswork? Evaluating Large Language Models' Medical Knowledge with Structured One-Hop Judgments [108.55277188617035]
  Large language models (LLMs) have been widely adopted in various downstream task domains, but their abilities to directly recall and apply factual medical knowledge remains under-explored.<n>We introduce the Medical Knowledge Judgment dataset (MKJ), a dataset derived from the Unified Medical Language System (UMLS), a comprehensive repository of standardized vocabularies and knowledge graphs.<n>Through a binary classification framework, MKJ evaluates LLMs' grasp of fundamental medical facts by having them assess the validity of concise, one-hop statements.
  arXiv  Detail & Related papers  (2025-02-20T05:27:51Z)
• The Decoy Dilemma in Online Medical Information Evaluation: A Comparative Study of Credibility Assessments by LLM and Human Judges [4.65004369765875]
  It is not clear to what extent large language models (LLMs) behave "rationally" Our study empirically confirms the cognitive bias risks embedded in LLM agents. It highlights the complexity and importance of debiasing AI agents.
  arXiv  Detail & Related papers  (2024-11-23T00:43:27Z)
• Persuasion with Large Language Models: a Survey [49.86930318312291]
  Large Language Models (LLMs) have created new disruptive possibilities for persuasive communication. In areas such as politics, marketing, public health, e-commerce, and charitable giving, such LLM Systems have already achieved human-level or even super-human persuasiveness. Our survey suggests that the current and future potential of LLM-based persuasion poses profound ethical and societal risks.
  arXiv  Detail & Related papers  (2024-11-11T10:05:52Z)
• Human-AI collectives produce the most accurate differential diagnoses [0.0]
  We show that hybrid collectives of physicians and large language models (LLMs) outperform both single physicians and physician collectives. Our approach highlights the potential for collective human and machine intelligence to improve accuracy in complex, open-ended domains like medical diagnostics.
  arXiv  Detail & Related papers  (2024-06-21T08:46:30Z)
• MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks [49.60689355674541]
  A rich literature in cognitive science has studied people's causal and moral intuitions. This work has revealed a number of factors that systematically influence people's judgments. We test whether large language models (LLMs) make causal and moral judgments about text-based scenarios that align with human participants.
  arXiv  Detail & Related papers  (2023-10-30T15:57:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.