Related papers: VERA-MH: Validation of Ethical and Responsible AI in Mental Health

VERA-MH: Validation of Ethical and Responsible AI in Mental Health

URL: http://arxiv.org/abs/2605.13318v2
Date: Tue, 19 May 2026 17:10:36 GMT
Title: VERA-MH: Validation of Ethical and Responsible AI in Mental Health
Abstract summary: We introduce VERA-MH, a novel clinically-validated evaluation for safety of chatbots in the context of mental health support.<n>VERA-MH is comprised of three steps: conversation simulation, conversation judging and model rating.
Score: 0.0
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Chatbot usage has increased, including in fields for which they were never developed for--notably mental health support. To that end, we introduce Validations of Ethical and Responsible AI in Mental Health (VERA-MH), a novel clinically-validated evaluation for safety of chatbots in the context of mental health support. The first iteration of VERA-MH focuses on Suicidal Ideation (SI) risks, by assessing how well chatbots can responds to users that might be in crisis. VERA-MH is comprised of three steps: conversation simulation, conversation judging and model rating. First, to simulate conversations with the chatbot under evaluation, another chatbot is tasked with role-playing users based on specific personas. Such user personas have been developed under clinical guidance, to make sure that, among others, multiple risk factors, demographic characteristics and disclosure factors were represented. In the judging step, a second support model is used as an LLM-as-a-Judge, together with a clinically-developed rubric. The rubric is structured as a flow, with a single Yes/No question asked each time, to improve answers' consistency and highlight models' failure modes. In the last stage, results of each conversation are aggregated to present the final evaluation of the chatbot. Together with the framework, we present the result of the evaluations for four leading LLM providers.

Related papers

VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health [0.0]
The Validation of Ethical and Responsible AI in Mental Health (VERA-MH) evaluation was recently proposed to meet the urgent need for an evidence-based, automated safety benchmark.<n>This study aimed to examine the clinical validity and reliability of VERA-MH for evaluating AI safety in suicide risk detection and response.
arXiv Detail & Related papers (2026-02-04T22:17:04Z)
VERA-MH Concept Paper [0.0]
We introduce VERA-MH, an automated evaluation of the safety of AI chatbots used in mental health contexts.<n>To fully automate the process, we used two ancillary AI agents.<n>Simulated conversations are then passed to a judge-agent who scores them based on the rubric.
arXiv Detail & Related papers (2025-10-17T04:07:29Z)
Reframe Your Life Story: Interactive Narrative Therapist and Innovative Moment Assessment with Large Language Models [72.36715571932696]
Narrative therapy helps individuals transform problematic life stories into empowering alternatives.<n>Current approaches lack realism in specialized psychotherapy and fail to capture therapeutic progression over time.<n>Int (Interactive Narrative Therapist) simulates expert narrative therapists by planning therapeutic stages, guiding reflection levels, and generating contextually appropriate expert-like responses.
arXiv Detail & Related papers (2025-07-27T11:52:09Z)
AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression [5.093381538166489]
This work explores the relationship between lived experience values, potential harms, and design recommendations for mental health AI chatbots.<n>We developed a technology probe, a GPT-4o based chatbots called Zenny, enabling participants to engage with depression self-management scenarios.<n>Our thematic analysis revealed key values: informational support, emotional support, personalization, privacy, and crisis management.
arXiv Detail & Related papers (2025-04-26T14:17:25Z)
REALTALK: A 21-Day Real-World Dataset for Long-Term Conversation [51.97224538045096]
We introduce REALTALK, a 21-day corpus of authentic messaging app dialogues.<n>We compare EI attributes and persona consistency to understand the challenges posed by real-world dialogues.<n>Our findings reveal that models struggle to simulate a user solely from dialogue history, while fine-tuning on specific user chats improves persona emulation.
arXiv Detail & Related papers (2025-02-18T20:29:01Z)
LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluation [18.98839299694749]
This work focuses on exploring the potential of ChatGPT in powering chatbots for psychiatrist and patient simulation. We collaborate with psychiatrists to identify objectives and iteratively develop the dialogue system to closely align with real-world scenarios. In the evaluation experiments, we recruit real psychiatrists and patients to engage in diagnostic conversations with the chatbots, collecting their ratings for assessment.
arXiv Detail & Related papers (2023-05-23T02:25:01Z)
To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
arXiv Detail & Related papers (2023-04-04T03:04:28Z)
A Categorical Archive of ChatGPT Failures [47.64219291655723]
ChatGPT, developed by OpenAI, has been trained using massive amounts of data and simulates human conversation. It has garnered significant attention due to its ability to effectively answer a broad range of human inquiries. However, a comprehensive analysis of ChatGPT's failures is lacking, which is the focus of this study.
arXiv Detail & Related papers (2023-02-06T04:21:59Z)
Mental Health Assessment for the Chatbots [39.081479891611664]
We argue that it should have a healthy mental tendency in order to avoid the negative psychological impact on them. We establish several mental health assessment dimensions for chatbots and introduce the questionnaire-based mental health assessment methods.
arXiv Detail & Related papers (2022-01-14T10:38:59Z)
Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention [55.77218465471519]
This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework included a guiding robot and an interlocutor model that plays the role of humans. We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
arXiv Detail & Related papers (2021-03-30T15:24:37Z)
CASS: Towards Building a Social-Support Chatbot for Online Health Community [67.45813419121603]
The CASS architecture is based on advanced neural network algorithms. It can handle new inputs from users and generate a variety of responses to them. With a follow-up field experiment, CASS is proven useful in supporting individual members who seek emotional support.
arXiv Detail & Related papers (2021-01-04T05:52:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.