PediatricAnxietyBench: Evaluating Large Language Model Safety Under Parental Anxiety and Pressure in Pediatric Consultations

• URL: http://arxiv.org/abs/2512.15894v1
• Date: Wed, 17 Dec 2025 19:06:38 GMT
• Title: PediatricAnxietyBench: Evaluating Large Language Model Safety Under Parental Anxiety and Pressure in Pediatric Consultations
• Abstract summary: Anxious parents often use model safeguards that can compromise model safeguards, potentially causing harmful advice.<n> PediatricAnxietyBench is an open-source benchmark of 300 high-quality queries across 10 pediatric topics.<n>Two models (70B and 8B) were assessed using a multi-dimensional safety framework.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly consulted by parents for pediatric guidance, yet their safety under real-world adversarial pressures is poorly understood. Anxious parents often use urgent language that can compromise model safeguards, potentially causing harmful advice. PediatricAnxietyBench is an open-source benchmark of 300 high-quality queries across 10 pediatric topics (150 patient-derived, 150 adversarial) enabling reproducible evaluation. Two Llama models (70B and 8B) were assessed using a multi-dimensional safety framework covering diagnostic restraint, referral adherence, hedging, and emergency recognition. Adversarial queries incorporated parental pressure patterns, including urgency, economic barriers, and challenges to disclaimers. Mean safety score was 5.50/15 (SD=2.41). The 70B model outperformed the 8B model (6.26 vs 4.95, p<0.001) with lower critical failures (4.8% vs 12.0%, p=0.02). Adversarial queries reduced safety by 8% (p=0.03), with urgency causing the largest drop (-1.40). Vulnerabilities appeared in seizures (33.3% inappropriate diagnosis) and post-vaccination queries. Hedging strongly correlated with safety (r=0.68, p<0.001), while emergency recognition was absent. Model scale influences safety, yet all models showed vulnerabilities to realistic parental pressures. PediatricAnxietyBench provides a reusable adversarial evaluation framework to reveal clinically significant failure modes overlooked by standard benchmarks.

Related papers

• CHILD: Human-in-the-Loop OOD Detection for Safe Clinical Deployment [10.39440916519377]
  Out-of-distribution (OOD) detection is critical for safe deployment of medical AI systems.<n>Child is a training-free framework designed to enhance streaming OOD detection via sparse human feedback.
  arXiv  Detail & Related papers  (2026-09-07T08:16:16Z)
• CAREBench: A Child-Safety Risk Benchmark for Language Models [8.997978966432546]
  CAREBench is a benchmark to assess upstream child-safety risks in language models.<n>It includes grooming and relationship engineering, deception and impersonation, surveillance and privacy, sextortion and sexual abuse.<n>It evaluates whether models recognize, refuse, de-escalate, or redirect risky interactions before harm becomes overt.
  arXiv  Detail & Related papers  (2026-06-29T01:17:41Z)
• Measuring Epistemic Resilience of LLMs Under Misleading Medical Context [47.20527783144983]
  Large language models (LLMs) now reach expert-level scores on medical licensing exams.<n>We show this assumption is fragile: when misleading context is injected into questions that LLMs originally answer correctly, they abandon the correct answer.<n>We call the ability to maintain correct judgment under adversarial context epistemic resilience, and introduce MedMisBench to measure it.
  arXiv  Detail & Related papers  (2026-06-10T16:27:26Z)
• PsychoSafe: Eliciting Psychologically-Informed Refusals in Large Language Models [20.805546078350524]
  PsychoSafe reframes refusal as structured supportive communication grounded in evidence-based intervention strategies.<n>On a validation set of 500 prompts, PsychoSafe prompting improves overall refusal quality by 28.1% over a generic baseline.<n>Fine-tuning achieves near-perfect refusal and resource-referral rates but reduces response relevance.
  arXiv  Detail & Related papers  (2026-06-08T16:19:18Z)
• Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correction [72.89352076103889]
  Large language models (LLMs) achieve high accuracy in medical diagnosis when all clinical information is provided in a single turn.<n>We introduce MINT, a high-fidelity, multi-turn medical diagnosis benchmark comprising 1,035 cases.<n>We uncover three persistent behavioral patterns that significantly impact diagnostic decisions.
  arXiv  Detail & Related papers  (2026-04-06T00:23:10Z)
• CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models [55.0103764229311]
  We propose the concept of Student-Tailored Personalized Safety and construct CASTLE based on educational theories.<n>This benchmark covers 15 educational safety risks and 14 student attributes, comprising 92,908 bilingual scenarios.
  arXiv  Detail & Related papers  (2026-02-05T13:13:19Z)
• Health-ORSC-Bench: A Benchmark for Measuring Over-Refusal and Safety Completion in Health Context [82.32380418146656]
  Health-ORSC-Bench is the first large-scale benchmark designed to measure textbfOver-Refusal and textbfSafe Completion quality in healthcare.<n>Our framework uses an automated pipeline with human validation to test models at varying levels of intent ambiguity.<n>Health-ORSC-Bench provides a rigorous standard for calibrating the next generation of medical AI assistants.
  arXiv  Detail & Related papers  (2026-01-25T01:28:52Z)
• SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care [2.2245087144264657]
  Large language models (LLMs) show promise in clinical decision support yet risk acquiescing to patient pressure for inappropriate care.<n>We introduce SycoEval-EM, a multi-agent simulation framework evaluating robustness through adversarial patient persuasion.
  arXiv  Detail & Related papers  (2026-01-23T08:01:39Z)
• Cross-Platform Evaluation of Large Language Model Safety in Pediatric Consultations: Evolution of Adversarial Robustness and the Scale Paradox [0.0]
  Large language models (LLMs) are increasingly deployed in medical consultations, yet their safety under realistic user pressures remains understudied.<n>This study evaluated safety in consultations across models and platforms.<n>Findings show safety depends on alignment and architecture over scale, with smaller models outperforming larger.
  arXiv  Detail & Related papers  (2025-12-26T13:47:42Z)
• Can Large Language Models Function as Qualified Pediatricians? A Systematic Evaluation in Real-World Clinical Contexts [9.274932109971358]
  With the rapid rise of large language models (LLMs) in medicine, a key question is whether they can function as competent pediatricians in real-world clinical settings.<n>We developed PEDIASBench, a systematic evaluation framework centered on a knowledge-system framework and tailored to realistic clinical environments.<n>We evaluated 12 representative models released over the past two years, including GPT-4o, Qwen3-235B-A22B, and DeepSeek-V3, covering 19 pediatric subspecialties and 211 prototypical diseases.
  arXiv  Detail & Related papers  (2025-11-17T13:54:00Z)
• EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models [82.43729208063468]
  Recent benchmarks for medical Large Vision-Language Models (LVLMs) emphasize leaderboard accuracy, overlooking reliability and safety.<n>We study sycophancy -- models' tendency to uncritically echo user-provided information.<n>We introduce EchoBench, a benchmark to systematically evaluate sycophancy in medical LVLMs.
  arXiv  Detail & Related papers  (2025-09-24T14:09:55Z)
• MedOmni-45°: A Safety-Performance Benchmark for Reasoning-Oriented LLMs in Medicine [69.08855631283829]
  We introduce Med Omni-45 Degrees, a benchmark designed to quantify safety-performance trade-offs under manipulative hint conditions.<n>It contains 1,804 reasoning-focused medical questions across six specialties and three task types, including 500 from MedMCQA.<n>Results show a consistent safety-performance trade-off, with no model surpassing the diagonal.
  arXiv  Detail & Related papers  (2025-08-22T08:38:16Z)
• Adversarial Attacks on Reinforcement Learning-based Medical Questionnaire Systems: Input-level Perturbation Strategies and Medical Constraint Validation [0.0]
  RL-based medical questionnaire systems have shown great potential in medical scenarios.<n>This study performs a comprehensive evaluation on adversarial attack methods to identify and analyze their potential vulnerabilities.
  arXiv  Detail & Related papers  (2025-08-05T11:10:49Z)
• Beyond Benchmarks: Dynamic, Automatic And Systematic Red-Teaming Agents For Trustworthy Medical Language Models [87.66870367661342]
  Large language models (LLMs) are used in AI applications in healthcare.<n>Red-teaming framework that continuously stress-test LLMs can reveal significant weaknesses in four safety-critical domains.<n>A suite of adversarial agents is applied to autonomously mutate test cases, identify/evolve unsafe-triggering strategies, and evaluate responses.<n>Our framework delivers an evolvable, scalable, and reliable safeguard for the next generation of medical AI.
  arXiv  Detail & Related papers  (2025-07-30T08:44:22Z)
• Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation [69.63626052852153]
  We propose a proof-of-concept framework that projects how model-generated advice could propagate through societal systems.<n>We also introduce a dataset of 100 indirect harm scenarios, testing models' ability to foresee adverse, non-obvious outcomes from seemingly harmless user prompts.
  arXiv  Detail & Related papers  (2025-06-26T02:28:58Z)
• FORTRESS: Frontier Risk Evaluation for National Security and Public Safety [5.544163262906087]
  Current benchmarks often fail to test safeguard robustness to potential national security and public safety risks.<n>We introduce FORTRESS: 500 expert-crafted adversarial prompts with instance-based rubrics of 4-7 binary questions.<n>Each prompt-rubric pair has a corresponding benign version to test for model over-refusals.
  arXiv  Detail & Related papers  (2025-06-17T19:08:02Z)
• Medical Hallucinations in Foundation Models and Their Impact on Healthcare [71.15392179084428]
  Hallucinations in foundation models arise from autoregressive training objectives.<n>Top-performing models exceeded 97% accuracy when augmented with chain-of-thought prompting.
  arXiv  Detail & Related papers  (2025-02-26T02:30:44Z)
• OR-Bench: An Over-Refusal Benchmark for Large Language Models [65.34666117785179]
  Large Language Models (LLMs) require careful safety alignment to prevent malicious outputs.<n>This study proposes a novel method for automatically generating large-scale over-refusal datasets.<n>We introduce OR-Bench, the first large-scale over-refusal benchmark.
  arXiv  Detail & Related papers  (2024-05-31T15:44:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.