"Am I A Good Therapist?" Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

• URL: http://arxiv.org/abs/2102.11265v1
• Date: Mon, 22 Feb 2021 18:52:52 GMT
• Title: "Am I A Good Therapist?" Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies
• Authors: Nikolaos Flemotomos, Victor R. Martinez, Zhuohao Chen, Karan Singla, Victor Ardulov, Raghuveer Peri, Derek D. Caperton, James Gibson, Michael J. Tanana, Panayiotis Georgiou, Jake Van Epps, Sarah P. Lord, Tad Hirsch, Zac E. Imel, David C. Atkins, Shrikanth Narayanan
• Abstract summary: We describe our platform and its performance, using a dataset of more than 5,000 recordings. Our system gives comprehensive feedback to the therapist, including information about the dynamics of the session. We are confident that a widespread use of automated psychotherapy rating tools in the near future will augment experts' capabilities.
• Score: 38.726068038788384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the growing prevalence of psychological interventions, it is vital to have measures which rate the effectiveness of psychological care, in order to assist in training, supervision, and quality assurance of services. Traditionally, quality assessment is addressed by human raters who evaluate recorded sessions along specific dimensions, often codified through constructs relevant to the approach and domain. This is however a cost-prohibitive and time-consuming method which leads to poor feasibility and limited use in real-world settings. To facilitate this process, we have developed an automated competency rating tool able to process the raw recorded audio of a session, analyzing who spoke when, what they said, and how the health professional used language to provide therapy. Focusing on a use case of a specific type of psychotherapy called Motivational Interviewing, our system gives comprehensive feedback to the therapist, including information about the dynamics of the session (e.g., therapist's vs. client's talking time), low-level psychological language descriptors (e.g., type of questions asked), as well as other high-level behavioral constructs (e.g., the extent to which the therapist understands the clients' perspective). We describe our platform and its performance, using a dataset of more than 5,000 recordings drawn from its deployment in a real-world clinical setting used to assist training of new therapists. We are confident that a widespread use of automated psychotherapy rating tools in the near future will augment experts' capabilities by providing an avenue for more effective training and skill improvement and will eventually lead to more positive clinical outcomes.

Related papers

• CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy [67.23830698947637]
  We propose a new benchmark, CBT-BENCH, for the systematic evaluation of cognitive behavioral therapy (CBT) assistance. We include three levels of tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of multiple-choice questions; II: Cognitive model understanding, with the tasks of cognitive distortion classification, primary core belief classification, and fine-grained core belief classification; III: Therapeutic response generation, with the task of generating responses to patient speech in CBT therapy sessions. Experimental results indicate that while LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios
  arXiv  Detail & Related papers  (2024-10-17T04:52:57Z)
• Therapy as an NLP Task: Psychologists' Comparison of LLMs and Human Peers in CBT [6.812247730094931]
  We investigate the potential and limitations of using large language models (LLMs) as providers of evidence-based therapy. We replicated publicly accessible mental health conversations rooted in Cognitive Behavioral Therapy (CBT) to compare session dynamics and counselor's CBT-based behaviors. Our findings show that the peer sessions are characterized by empathy, small talk, therapeutic alliance, and shared experiences but often exhibit therapist drift.
  arXiv  Detail & Related papers  (2024-09-03T19:19:13Z)
• Enhancing AI-Driven Psychological Consultation: Layered Prompts with Large Language Models [44.99833362998488]
  We explore the use of large language models (LLMs) like GPT-4 to augment psychological consultation services. Our approach introduces a novel layered prompting system that dynamically adapts to user input. We also develop empathy-driven and scenario-based prompts to enhance the LLM's emotional intelligence.
  arXiv  Detail & Related papers  (2024-08-29T05:47:14Z)
• Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy [6.952909762512736]
  We examine the effects of prompt engineering to guide Large Language Models (LLMs) in delivering parts of a Problem-Solving Therapy session via text. We demonstrate that the models' capability to deliver protocolized therapy can be improved with the proper use of prompt engineering methods.
  arXiv  Detail & Related papers  (2024-08-27T17:25:16Z)
• PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agents [68.50571379012621]
  Psychological measurement is essential for mental health, self-understanding, and personal development. PsychoGAT (Psychological Game AgenTs) achieves statistically significant excellence in psychometric metrics such as reliability, convergent validity, and discriminant validity.
  arXiv  Detail & Related papers  (2024-02-19T18:00:30Z)
• Evaluating the Efficacy of Interactive Language Therapy Based on LLM for High-Functioning Autistic Adolescent Psychological Counseling [1.1780706927049207]
  This study investigates the efficacy of Large Language Models (LLMs) in interactive language therapy for high-functioning autistic adolescents. LLMs present a novel opportunity to augment traditional psychological counseling methods.
  arXiv  Detail & Related papers  (2023-11-12T07:55:39Z)
• Enhancing Psychological Counseling with Large Language Model: A Multifaceted Decision-Support System for Non-Professionals [31.01304974679576]
  This paper introduces a novel model constructed on the foundation of large language models to assist non-professionals in providing psychological interventions on online user discourses. A comprehensive study was conducted involving ten professional psychological counselors of varying expertise, evaluating the system. The findings affirm that our system is capable of analyzing patients' issues with relative accuracy and proffering professional-level strategies recommendations.
  arXiv  Detail & Related papers  (2023-08-29T10:20:53Z)
• Automated Quality Assessment of Cognitive Behavioral Therapy Sessions Through Highly Contextualized Language Representations [34.670548892766625]
  A BERT-based model is proposed for automatic behavioral scoring of a specific type of psychotherapy, called Cognitive Behavioral Therapy (CBT) The model is trained in a multi-task manner in order to achieve higher interpretability. BERT-based representations are further augmented with available therapy metadata, providing relevant non-linguistic context and leading to consistent performance improvements.
  arXiv  Detail & Related papers  (2021-02-23T09:22:29Z)
• Designing Personalized Interaction of a Socially Assistive Robot for Stroke Rehabilitation Therapy [64.52563354823711]
  The research of a socially assistive robot has a potential to augment and assist physical therapy sessions for patients with neurological and musculoskeletal problems. This paper presents an interactive approach of a socially assistive robot that can dynamically select kinematic features of assessment on individual patient's exercises to predict the quality of motion.
  arXiv  Detail & Related papers  (2020-07-13T16:12:05Z)
• Opportunities of a Machine Learning-based Decision Support System for Stroke Rehabilitation Assessment [64.52563354823711]
  Rehabilitation assessment is critical to determine an adequate intervention for a patient. Current practices of assessment mainly rely on therapist's experience, and assessment is infrequently executed due to the limited availability of a therapist. We developed an intelligent decision support system that can identify salient features of assessment using reinforcement learning.
  arXiv  Detail & Related papers  (2020-02-27T17:04:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.