Therapy as an NLP Task: Psychologists' Comparison of LLMs and Human Peers in CBT

• URL: http://arxiv.org/abs/2409.02244v1
• Date: Tue, 3 Sep 2024 19:19:13 GMT
• Title: Therapy as an NLP Task: Psychologists' Comparison of LLMs and Human Peers in CBT
• Authors: Zainab Iftikhar, Sean Ransom, Amy Xiao, Jeff Huang,
• Abstract summary: We investigate the potential and limitations of using large language models (LLMs) as providers of evidence-based therapy. We replicated publicly accessible mental health conversations rooted in Cognitive Behavioral Therapy (CBT) to compare session dynamics and counselor's CBT-based behaviors. Our findings show that the peer sessions are characterized by empathy, small talk, therapeutic alliance, and shared experiences but often exhibit therapist drift.
• Score: 6.812247730094931
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Wider access to therapeutic care is one of the biggest challenges in mental health treatment. Due to institutional barriers, some people seeking mental health support have turned to large language models (LLMs) for personalized therapy, even though these models are largely unsanctioned and untested. We investigate the potential and limitations of using LLMs as providers of evidence-based therapy by using mixed methods clinical metrics. Using HELPERT, a prompt run on a large language model using the same process and training as a comparative group of peer counselors, we replicated publicly accessible mental health conversations rooted in Cognitive Behavioral Therapy (CBT) to compare session dynamics and counselor's CBT-based behaviors between original peer support sessions and their reconstructed HELPERT sessions. Two licensed, CBT-trained clinical psychologists evaluated the sessions using the Cognitive Therapy Rating Scale and provided qualitative feedback. Our findings show that the peer sessions are characterized by empathy, small talk, therapeutic alliance, and shared experiences but often exhibit therapist drift. Conversely, HELPERT reconstructed sessions exhibit minimal therapist drift and higher adherence to CBT methods but display a lack of collaboration, empathy, and cultural understanding. Through CTRS ratings and psychologists' feedback, we highlight the importance of human-AI collaboration for scalable mental health. Our work outlines the ethical implication of imparting human-like subjective qualities to LLMs in therapeutic settings, particularly the risk of deceptive empathy, which may lead to unrealistic patient expectations and potential harm.

Related papers

• CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy [67.23830698947637]
  We propose a new benchmark, CBT-BENCH, for the systematic evaluation of cognitive behavioral therapy (CBT) assistance. We include three levels of tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of multiple-choice questions; II: Cognitive model understanding, with the tasks of cognitive distortion classification, primary core belief classification, and fine-grained core belief classification; III: Therapeutic response generation, with the task of generating responses to patient speech in CBT therapy sessions. Experimental results indicate that while LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios
  arXiv  Detail & Related papers  (2024-10-17T04:52:57Z)
• Are Large Language Models Possible to Conduct Cognitive Behavioral Therapy? [13.0263170692984]
  Large language models (LLMs) have been validated, providing new possibilities for psychological assistance therapy. Many concerns have been raised by mental health experts regarding the use of LLMs for therapy. Four LLM variants with excellent performance on natural language processing are evaluated.
  arXiv  Detail & Related papers  (2024-07-25T03:01:47Z)
• Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theory [24.937025825501998]
  We create a multi-turn dialogue dataset that emulates real-life interactions using the goal-oriented and structured approach of Cognitive Behavioral Therapy (CBT) We benchmark against established psychological criteria used to evaluate real counseling sessions, ensuring alignment with expert evaluations. Experimental results demonstrate that Camel, a model trained with Cactus, outperforms other models in counseling skills, highlighting its effectiveness and potential as a counseling agent.
  arXiv  Detail & Related papers  (2024-07-03T13:41:31Z)
• HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy [25.908522131646258]
  We unveil the Helping and Empowering through Adaptive Language in Mental Enhancement (HealMe) model. This novel cognitive reframing therapy method effectively addresses deep-rooted negative thoughts and fosters rational, balanced perspectives. We adopt the first comprehensive and expertly crafted psychological evaluation metrics, specifically designed to rigorously assess the performance of cognitive reframing.
  arXiv  Detail & Related papers  (2024-02-26T09:10:34Z)
• PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents [68.50571379012621]
  Psychological measurement is essential for mental health, self-understanding, and personal development. PsychoGAT (Psychological Game AgenTs) achieves statistically significant excellence in psychometric metrics such as reliability, convergent validity, and discriminant validity.
  arXiv  Detail & Related papers  (2024-02-19T18:00:30Z)
• Chain of Empathy: Enhancing Empathetic Response of Large Language Models Based on Psychotherapy Models [2.679689033125693]
  We present a novel method, the Chain of Empathy (CoE) prompting, that utilizes insights from psychotherapy to induce Large Language Models (LLMs) to reason about human emotional states. This method is inspired by various psychotherapy approaches including Cognitive Behavioral Therapy (CBT), Dialectical Behavior Therapy (DBT), Person Centered Therapy (PCT), and Reality Therapy (RT)
  arXiv  Detail & Related papers  (2023-11-02T02:21:39Z)
• Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing [53.096237570992294]
  Strategy training is a rehabilitation approach that teaches skills to reduce disability among those with cognitive impairments following a stroke. Standardized fidelity assessment is used to measure adherence to treatment principles. We developed a rule-based NLP algorithm, a long-short term memory (LSTM) model, and a bidirectional encoder representation from transformers (BERT) model for this task.
  arXiv  Detail & Related papers  (2022-09-14T15:33:30Z)
• Enabling AI and Robotic Coaches for Physical Rehabilitation Therapy: Iterative Design and Evaluation with Therapists and Post-Stroke Survivors [66.07833535962762]
  Artificial intelligence (AI) and robotic coaches promise the improved engagement of patients on rehabilitation exercises through social interaction. Previous work explored the potential of automatically monitoring exercises for AI and robotic coaches, but deployment remains a challenge. We present our efforts on eliciting the detailed design specifications on how AI and robotic coaches could interact with and guide patient's exercises.
  arXiv  Detail & Related papers  (2021-06-15T22:06:39Z)
• STAN: A stuttering therapy analysis helper [59.37911277681339]
  Stuttering is a complex speech disorder identified by repeti-tions, prolongations of sounds, syllables or words and blockswhile speaking. We introduceSTAN, a system to aid speech therapists in stuttering therapysessions.
  arXiv  Detail & Related papers  (2021-06-15T13:48:12Z)
• MET: Multimodal Perception of Engagement for Telehealth [52.54282887530756]
  We present MET, a learning-based algorithm for perceiving a human's level of engagement from videos. We release a new dataset, MEDICA, for mental health patient engagement detection.
  arXiv  Detail & Related papers  (2020-11-17T15:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.