MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models

• URL: http://arxiv.org/abs/2604.17730v1
• Date: Mon, 20 Apr 2026 02:37:45 GMT
• Title: MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models
• Abstract summary: Large language models (LLMs) are increasingly explored as scalable tools for mental health counseling.<n> evaluating their safety remains challenging due to the interactional and context-dependent nature of clinical harm.<n>We introduce R-MHSafe, a role-aware mental health safety taxonomy, and MHSafeEval, a closed-loop, agent-based evaluation framework.
• Score: 32.31200693426663
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly explored as scalable tools for mental health counseling, yet evaluating their safety remains challenging due to the interactional and context-dependent nature of clinical harm. Existing evaluation frameworks predominantly assess isolated responses using coarse-grained taxonomies or static datasets, limiting their ability to diagnose how harms emerge and accumulate over multi-turn counseling interactions. In this work, we introduce R-MHSafe, a role-aware mental health safety taxonomy that characterizes clinically significant harm in terms of the interactional roles an AI counselor adopts, including perpetrator, instigator, facilitator, or enabler, combined with clinically grounded harm categories. Then, we propose MHSafeEval, a closed-loop, agent-based evaluation framework that formulates safety assessment as trajectory-level discovery of harm through adversarial multi-turn interactions, guided by role-aware modeling. Using R-MHSafe and MHSafeEval, we conduct a large-scale evaluation across state-of-the-art LLMs. Our results reveal substantial role-dependent and cumulative safety failures that are systematically missed by existing static benchmarks, and show that our framework significantly improves failure-mode coverage and diagnostic granularity.

Related papers

• Retrieval-Augmented Generation in LLMs for Mental Health: Quantifying the Incremental Contribution of Retrieval Within a Layered Safety Architecture [1.9213555198905314]
  Digital mental health interventions (DMHIs) offer scalable support, but ensuring they accurately detect users' intent during volatile situations can be challenging.<n>This paper evaluates six Large Language models within a DMHI called Wysa, via a controlled comparison of RAG-enabled versus RAG-disabled modes.<n>The study computed classification accuracy, recall, precision and F1 scores against ground truth labels and tested differences for statistical significance.
  arXiv  Detail & Related papers  (2026-07-17T06:32:12Z)
• Towards Understanding and Measuring COGNITIVE ATROPHY in LLM Behaviour [10.419215828520414]
  Existing benchmarks measure knowledge, safety, or static response quality.<n>We formalize this missing dimension as COGNITIVE ATROPHY, a process-level behavioural measure in AI-mediated mental-health support.
  arXiv  Detail & Related papers  (2026-06-16T16:26:12Z)
• CareGuardAI: Context-Aware Multi-Agent Guardrails for Clinical Safety & Hallucination Mitigation in Patient-Facing LLMs [5.437368195729715]
  CareGuardAI is a risk-aware safety framework for patient-facing medical question answering.<n>At inference time, CareGuardAI employs a multi-stage pipeline consisting of a controller agent, safety-constrained generation, and dual risk evaluation.<n>We evaluate CareGuardAI on PatientSafeBench, MedSafetyBench, and MedHallu, covering both safety and hallucination detection.
  arXiv  Detail & Related papers  (2026-04-07T10:54:57Z)
• SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond [134.43113804188195]
  We introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts.<n>SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement.
  arXiv  Detail & Related papers  (2026-03-02T08:16:04Z)
• VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health [0.0]
  The Validation of Ethical and Responsible AI in Mental Health (VERA-MH) evaluation was recently proposed to meet the urgent need for an evidence-based, automated safety benchmark.<n>This study aimed to examine the clinical validity and reliability of VERA-MH for evaluating AI safety in suicide risk detection and response.
  arXiv  Detail & Related papers  (2026-02-04T22:17:04Z)
• MHDash: An Online Platform for Benchmarking Mental Health-Aware AI Assistants [2.89303424493]
  We present MHDash, an open-source platform designed to support the development, evaluation, and auditing of AI systems for mental health applications.<n>Our results reveal several key findings, including that simple baselines and advanced LLM APIs exhibit comparable overall accuracy yet diverge significantly on high-risk cases.<n>By releasing MHDash as an open platform, we aim to promote reproducible research, transparent evaluation, and safety-aligned development of AI systems for mental health support.
  arXiv  Detail & Related papers  (2026-01-30T22:03:31Z)
• SafeRBench: A Comprehensive Benchmark for Safety Assessment in Large Reasoning Models [60.8821834954637]
  We present SafeRBench, the first benchmark that assesses LRM safety end-to-end.<n>We pioneer the incorporation of risk categories and levels into input design.<n>We introduce a micro-thought chunking mechanism to segment long reasoning traces into semantically coherent units.
  arXiv  Detail & Related papers  (2025-11-19T06:46:33Z)
• Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs [6.0460961868478975]
  We introduce a unified taxonomy of six clinically-informed mental health crisis categories.<n>We benchmark three state-of-the-art LLMs for their ability to classify crisis types and generate safe, appropriate responses.<n>We identify systemic weaknesses in handling indirect or ambiguous risk signals, a reliance on formulaic and inauthentic default replies, and frequent misalignment with user context.
  arXiv  Detail & Related papers  (2025-09-29T14:42:23Z)
• Psychometric Personality Shaping Modulates Capabilities and Safety in Language Models [3.9481669393262675]
  We investigate how psychometric personality control grounded in the Big Five framework influences AI behavior in the context of capability and safety benchmarks.<n>Our experiments reveal striking effects: for example, reducing conscientiousness leads to significant drops in safety-relevant metrics on benchmarks such as WMDP, TruthfulQA, ETHICS, and Sycophancy.<n>These findings highlight personality shaping as a powerful and underexplored axis of model control that interacts with both safety and general competence.
  arXiv  Detail & Related papers  (2025-09-19T18:19:56Z)
• Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation [69.63626052852153]
  We propose a proof-of-concept framework that projects how model-generated advice could propagate through societal systems.<n>We also introduce a dataset of 100 indirect harm scenarios, testing models' ability to foresee adverse, non-obvious outcomes from seemingly harmless user prompts.
  arXiv  Detail & Related papers  (2025-06-26T02:28:58Z)
• The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs [57.1838332916627]
  Large Language Models (LLMs) have shown remarkable capabilities in Natural Language Processing (NLP)<n>Their widespread deployment has also raised significant safety concerns.<n>LLMs-generated content can exhibit unsafe behaviors such as toxicity, bias, or misinformation, especially in adversarial contexts.
  arXiv  Detail & Related papers  (2025-06-06T05:50:50Z)
• Multimodal Situational Safety [73.63981779844916]
  We present the first evaluation and analysis of a novel safety challenge termed Multimodal Situational Safety.<n>For an MLLM to respond safely, whether through language or action, it often needs to assess the safety implications of a language query within its corresponding visual context.<n>We develop the Multimodal Situational Safety benchmark (MSSBench) to assess the situational safety performance of current MLLMs.
  arXiv  Detail & Related papers  (2024-10-08T16:16:07Z)
• The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness [56.174255970895466]
  Large Language Models (LLMs) play an increasingly pivotal role in natural language processing applications. This paper presents Safety and Over-Defensiveness Evaluation (SODE) benchmark.
  arXiv  Detail & Related papers  (2023-12-30T17:37:06Z)
• A Benchmark for Understanding Dialogue Safety in Mental Health Support [15.22008156903607]
  This paper aims to develop a theoretically and factually grounded taxonomy that prioritizes the positive impact on help-seekers. We analyze the dataset using popular language models, including BERT-base, RoBERTa-large, and ChatGPT. The developed dataset and findings serve as valuable benchmarks for advancing research on dialogue safety in mental health support.
  arXiv  Detail & Related papers  (2023-07-31T07:33:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.