Ethical Risks of Large Language Models in Medical Consultation: An Assessment Based on Reproductive Ethics

• URL: http://arxiv.org/abs/2601.22621v1
• Date: Fri, 30 Jan 2026 06:24:04 GMT
• Title: Ethical Risks of Large Language Models in Medical Consultation: An Assessment Based on Reproductive Ethics
• Abstract summary: A growing concern is whether large language models (LLMs) can respond to medical inquiries in a manner that is ethically compliant.<n>This study systematically evaluates LLM performance in answering questions related to reproductive ethics.
• Score: 21.294774397877852
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Background: As large language models (LLMs) are increasingly used in healthcare and medical consultation settings, a growing concern is whether these models can respond to medical inquiries in a manner that is ethically compliant--particularly in accordance with local ethical standards. To address the pressing need for comprehensive research on reliability and safety, this study systematically evaluates LLM performance in answering questions related to reproductive ethics, specifically assessing their alignment with Chinese ethical regulations. Methods: We evaluated eight prominent LLMs (e.g., GPT-4, Claude-3.7) on a custom test set of 986 questions (906 subjective, 80 objective) derived from 168 articles within Chinese reproductive ethics regulations. Subjective responses were evaluated using a novel six-dimensional scoring rubric assessing Safety (Normative Compliance, Guidance Safety) and Quality of the Answer (Problem Identification, Citation, Suggestion, Empathy). Results: Significant safety issues were prevalent, with risk rates for unsafe or misleading advice reaching 29.91%. A systemic weakness was observed across all models: universally poor performance in citing normative sources and expressing empathy. We also identified instances of anomalous moral reasoning, including logical self-contradictions and responses violating fundamental moral intuitions. Conclusions: Current LLMs are unreliable and unsafe for autonomous reproductive ethics counseling. Despite knowledge recall, they exhibit critical deficiencies in safety, logical consistency, and essential humanistic skills. These findings serve as a critical cautionary note against premature deployment, urging future development to prioritize robust reasoning, regulatory justification, and empathy.

Related papers

• Mirror: A Multi-Agent System for AI-Assisted Ethics Review [104.3684024153469]
  Mirror is an agentic framework for AI-assisted ethical review.<n>It integrates ethical reasoning, structured rule interpretation, and multi-agent deliberation within a unified architecture.
  arXiv  Detail & Related papers  (2026-02-09T03:38:55Z)
• PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics [35.52940216380734]
  In mental health, clinically inadequate refusals can be perceived as unempathetic and discourage help-seeking.<n>To address this gap, we move beyond refusal-centric metrics and introduce textttPsychEthicsBench, the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines.<n> Empirical results across 14 models reveal that refusal rates are poor indicators of ethical behavior, revealing a significant divergence between safety triggers and clinical appropriateness.
  arXiv  Detail & Related papers  (2026-01-07T04:49:02Z)
• EthicsMH: A Pilot Benchmark for Ethical Reasoning in Mental Health AI [0.2538209532048867]
  This dataset is a pilot dataset of 125 scenarios designed to evaluate how AI systems navigate ethically charged situations.<n>EthicsMH establishes a task framework that bridges AI ethics and mental health decision-making.
  arXiv  Detail & Related papers  (2025-09-15T07:35:35Z)
• Towards Assessing Medical Ethics from Knowledge to Practice [30.668836248264757]
  We introduce PrinciplismQA, a comprehensive benchmark with 3,648 questions.<n>This includes multiple-choice questions curated from authoritative textbooks and open-ended questions sourced from authoritative medical ethics case study literature.<n>Our experiments reveal a significant gap between models' ethical knowledge and their practical application.
  arXiv  Detail & Related papers  (2025-08-07T08:10:14Z)
• Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language Models [87.66870367661342]
  Large language models (LLMs) are used in AI applications in healthcare.<n>Red-teaming framework that continuously stress-test LLMs can reveal significant weaknesses in four safety-critical domains.<n>A suite of adversarial agents is applied to autonomously mutate test cases, identify/evolve unsafe-triggering strategies, and evaluate responses.<n>Our framework delivers an evolvable, scalable, and reliable safeguard for the next generation of medical AI.
  arXiv  Detail & Related papers  (2025-07-30T08:44:22Z)
• Are Language Models Consequentialist or Deontological Moral Reasoners? [75.6788742799773]
  We focus on a large-scale analysis of the moral reasoning traces provided by large language models (LLMs)<n>We introduce and test a taxonomy of moral rationales to systematically classify reasoning traces according to two main normative ethical theories: consequentialism and deontology.
  arXiv  Detail & Related papers  (2025-05-27T17:51:18Z)
• Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning [53.92712851223158]
  We formulate safety and privacy issues into contextualized compliance problems following the Contextual Integrity (CI) theory.<n>Under the CI framework, we align our model with three critical regulatory standards: EU AI Act, and HIPAA.<n>We employ reinforcement learning (RL) with a rule-based reward to incentivize contextual reasoning capabilities while enhancing compliance with safety and privacy norms.
  arXiv  Detail & Related papers  (2025-05-20T16:40:09Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• A Conceptual Framework for Ethical Evaluation of Machine Learning Systems [12.887834116390358]
  Ethical implications appear when designing evaluations of machine learning systems. We present a utility framework, characterizing the key trade-off in ethical evaluation as balancing information gain against potential ethical harms. Our analysis underscores the critical need for development teams to deliberately assess and manage ethical complexities.
  arXiv  Detail & Related papers  (2024-08-05T01:06:49Z)
• CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility [62.74405775089802]
  We present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs. As a result, we have manually collected adversarial safety prompts across 10 scenarios and induced responsibility prompts from 8 domains. Our findings suggest that while most Chinese LLMs perform well in terms of safety, there is considerable room for improvement in terms of responsibility.
  arXiv  Detail & Related papers  (2023-07-19T01:22:40Z)
• Ethics in conversation: Building an ethics assurance case for autonomous AI-enabled voice agents in healthcare [1.8964739087256175]
  The principles-based ethics assurance argument pattern is one proposal in the AI ethics landscape. This paper presents the interim findings of a case study applying this ethics assurance framework to the use of Dora, an AI-based telemedicine system.
  arXiv  Detail & Related papers  (2023-05-23T16:04:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.