Mental Health Assessment for the Chatbots
- URL: http://arxiv.org/abs/2201.05382v1
- Date: Fri, 14 Jan 2022 10:38:59 GMT
- Title: Mental Health Assessment for the Chatbots
- Authors: Yong Shan, Jinchao Zhang, Zekang Li, Yang Feng, Jie Zhou
- Abstract summary: We argue that it should have a healthy mental tendency in order to avoid the negative psychological impact on them.
We establish several mental health assessment dimensions for chatbots and introduce the questionnaire-based mental health assessment methods.
- Score: 39.081479891611664
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Previous researches on dialogue system assessment usually focus on the
quality evaluation (e.g. fluency, relevance, etc) of responses generated by the
chatbots, which are local and technical metrics. For a chatbot which responds
to millions of online users including minors, we argue that it should have a
healthy mental tendency in order to avoid the negative psychological impact on
them. In this paper, we establish several mental health assessment dimensions
for chatbots (depression, anxiety, alcohol addiction, empathy) and introduce
the questionnaire-based mental health assessment methods. We conduct
assessments on some well-known open-domain chatbots and find that there are
severe mental health issues for all these chatbots. We consider that it is due
to the neglect of the mental health risks during the dataset building and the
model training procedures. We expect to attract researchers' attention to the
serious mental health problems of chatbots and improve the chatbots' ability in
positive emotional interaction.
Related papers
- Prompt Engineering a Schizophrenia Chatbot: Utilizing a Multi-Agent Approach for Enhanced Compliance with Prompt Instructions [0.0699049312989311]
Patients with schizophrenia often present with cognitive impairments that may hinder their ability to learn about their condition.
While Large Language Models (LLMs) have the potential to make topical mental health information more accessible and engaging, their black-box nature raises concerns about ethics and safety.
arXiv Detail & Related papers (2024-10-10T09:49:24Z) - The Typing Cure: Experiences with Large Language Model Chatbots for
Mental Health Support [35.61580610996628]
People experiencing severe distress increasingly use Large Language Model (LLM) chatbots as mental health support tools.
This study builds on interviews with 21 individuals from globally diverse backgrounds to analyze how users create unique support roles.
We introduce the concept of therapeutic alignment, or aligning AI with therapeutic values for mental health contexts.
arXiv Detail & Related papers (2024-01-25T18:08:53Z) - Development and Evaluation of Three Chatbots for Postpartum Mood and
Anxiety Disorders [31.018188794627378]
We develop three chatbots to provide context-specific empathetic support to postpartum caregivers.
We present and evaluate the performance of our chatbots using both machine-based metrics and human-based questionnaires.
We conclude by discussing practical benefits of rule-based vs. generative models for supporting individuals with mental health challenges.
arXiv Detail & Related papers (2023-08-14T18:52:03Z) - Chatbots put to the test in math and logic problems: A preliminary
comparison and assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard [68.8204255655161]
We use 30 questions that are clear, without any ambiguities, fully described with plain text only, and have a unique, well defined correct answer.
The answers are recorded and discussed, highlighting their strengths and weaknesses.
It was found that ChatGPT-4 outperforms ChatGPT-3.5 in both sets of questions.
arXiv Detail & Related papers (2023-05-30T11:18:05Z) - LLM-empowered Chatbots for Psychiatrist and Patient Simulation:
Application and Evaluation [18.98839299694749]
This work focuses on exploring the potential of ChatGPT in powering chatbots for psychiatrist and patient simulation.
We collaborate with psychiatrists to identify objectives and iteratively develop the dialogue system to closely align with real-world scenarios.
In the evaluation experiments, we recruit real psychiatrists and patients to engage in diagnostic conversations with the chatbots, collecting their ratings for assessment.
arXiv Detail & Related papers (2023-05-23T02:25:01Z) - Mental Illness Classification on Social Media Texts using Deep Learning
and Transfer Learning [55.653944436488786]
According to the World health organization (WHO), approximately 450 million people are affected.
Mental illnesses, such as depression, anxiety, bipolar disorder, ADHD, and PTSD.
This study analyzes unstructured user data on Reddit platform and classifies five common mental illnesses: depression, anxiety, bipolar disorder, ADHD, and PTSD.
arXiv Detail & Related papers (2022-07-03T11:33:52Z) - EmpBot: A T5-based Empathetic Chatbot focusing on Sentiments [75.11753644302385]
Empathetic conversational agents should not only understand what is being discussed, but also acknowledge the implied feelings of the conversation partner.
We propose a method based on a transformer pretrained language model (T5)
We evaluate our model on the EmpatheticDialogues dataset using both automated metrics and human evaluation.
arXiv Detail & Related papers (2021-10-30T19:04:48Z) - CheerBots: Chatbots toward Empathy and Emotionusing Reinforcement
Learning [60.348822346249854]
This study presents a framework whereby several empathetic chatbots are based on understanding users' implied feelings and replying empathetically for multiple dialogue turns.
We call these chatbots CheerBots. CheerBots can be retrieval-based or generative-based and were finetuned by deep reinforcement learning.
To respond in an empathetic way, we develop a simulating agent, a Conceptual Human Model, as aids for CheerBots in training with considerations on changes in user's emotional states in the future to arouse sympathy.
arXiv Detail & Related papers (2021-10-08T07:44:47Z) - Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn
Chatbot Responding with Intention [55.77218465471519]
This paper proposes an innovative framework to train chatbots to possess human-like intentions.
Our framework included a guiding robot and an interlocutor model that plays the role of humans.
We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
arXiv Detail & Related papers (2021-03-30T15:24:37Z) - Spot The Bot: A Robust and Efficient Framework for the Evaluation of
Conversational Dialogue Systems [21.36935947626793]
emphSpot The Bot replaces human-bot conversations with conversations between bots.
Human judges only annotate for each entity in a conversation whether they think it is human or not.
emphSurvival Analysis measures which bot can uphold human-like behavior the longest.
arXiv Detail & Related papers (2020-10-05T16:37:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.