CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling

• URL: http://arxiv.org/abs/2405.16433v3
• Date: Mon, 10 Jun 2024 11:43:48 GMT
• Title: CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling
• Authors: Chenhao Zhang, Renhao Li, Minghuan Tan, Min Yang, Jingwei Zhu, Di Yang, Jiahao Zhao, Guancheng Ye, Chengming Li, Xiping Hu,
• Abstract summary: We propose CPsyCoun, a report-based multi-turn dialogue reconstruction and evaluation framework for Chinese psychological counseling. To fully exploit psychological counseling reports, a two-phase approach is devised to construct high-quality dialogues. A comprehensive evaluation benchmark is developed for the effective automatic evaluation of multi-turn psychological consultations.
• Score: 27.193022503592342
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Using large language models (LLMs) to assist psychological counseling is a significant but challenging task at present. Attempts have been made on improving empathetic conversations or acting as effective assistants in the treatment with LLMs. However, the existing datasets lack consulting knowledge, resulting in LLMs lacking professional consulting competence. Moreover, how to automatically evaluate multi-turn dialogues within the counseling process remains an understudied area. To bridge the gap, we propose CPsyCoun, a report-based multi-turn dialogue reconstruction and evaluation framework for Chinese psychological counseling. To fully exploit psychological counseling reports, a two-phase approach is devised to construct high-quality dialogues while a comprehensive evaluation benchmark is developed for the effective automatic evaluation of multi-turn psychological consultations. Competitive experimental results demonstrate the effectiveness of our proposed framework in psychological counseling. We open-source the datasets and model for future research at https://github.com/CAS-SIAT-XinHai/CPsyCoun

Related papers

• CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy [67.23830698947637]
  We propose a new benchmark, CBT-BENCH, for the systematic evaluation of cognitive behavioral therapy (CBT) assistance. We include three levels of tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of multiple-choice questions; II: Cognitive model understanding, with the tasks of cognitive distortion classification, primary core belief classification, and fine-grained core belief classification; III: Therapeutic response generation, with the task of generating responses to patient speech in CBT therapy sessions. Experimental results indicate that while LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios
  arXiv  Detail & Related papers  (2024-10-17T04:52:57Z)
• Enhancing AI-Driven Psychological Consultation: Layered Prompts with Large Language Models [44.99833362998488]
  We explore the use of large language models (LLMs) like GPT-4 to augment psychological consultation services. Our approach introduces a novel layered prompting system that dynamically adapts to user input. We also develop empathy-driven and scenario-based prompts to enhance the LLM's emotional intelligence.
  arXiv  Detail & Related papers  (2024-08-29T05:47:14Z)
• Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions [12.455050661682051]
  We propose a framework that employs two large language models (LLMs) via role-playing for simulating counselor-client interactions. Our framework involves two LLMs, one acting as a client equipped with a specific and real-life user profile and the other playing the role of an experienced counselor.
  arXiv  Detail & Related papers  (2024-08-28T13:29:59Z)
• PsycoLLM: Enhancing LLM for Psychological Understanding and Evaluation [27.575675130769437]
  We propose a specialized psychological large language model (LLM), named PsycoLLM, trained on a proposed high-quality psychological dataset. We construct multi-turn dialogues through a three-step pipeline comprising generation, evidence judgment, and refinement. To compare the performance of PsycoLLM with other LLMs, we develop a comprehensive psychological benchmark based on authoritative psychological counseling examinations in China.
  arXiv  Detail & Related papers  (2024-07-08T08:25:56Z)
• Cactus: Towards Psychological Counseling Conversations using Cognitive Behavioral Theory [24.937025825501998]
  We create a multi-turn dialogue dataset that emulates real-life interactions using the goal-oriented and structured approach of Cognitive Behavioral Therapy (CBT) We benchmark against established psychological criteria used to evaluate real counseling sessions, ensuring alignment with expert evaluations. Experimental results demonstrate that Camel, a model trained with Cactus, outperforms other models in counseling skills, highlighting its effectiveness and potential as a counseling agent.
  arXiv  Detail & Related papers  (2024-07-03T13:41:31Z)
• Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting [46.919537239016734]
  Large language models (LLMs) have simplified the implementation of multi-turn dialogues. It remains challenging to deliver satisfactory performance in low-resource domain, like psychological dialogue dialogue. We propose a knowledge-driven progressive thought prompting method to guide LLM to generate psychology-related dialogue.
  arXiv  Detail & Related papers  (2024-06-24T12:02:56Z)
• LLM Questionnaire Completion for Automatic Psychiatric Assessment [49.1574468325115]
  We employ a Large Language Model (LLM) to convert unstructured psychological interviews into structured questionnaires spanning various psychiatric and personality domains. The obtained answers are coded as features, which are used to predict standardized psychiatric measures of depression (PHQ-8) and PTSD (PCL-C)
  arXiv  Detail & Related papers  (2024-06-09T09:03:11Z)
• Can Large Language Models be Used to Provide Psychological Counselling? An Analysis of GPT-4-Generated Responses Using Role-play Dialogues [0.0]
  Mental health care poses an increasingly serious challenge to modern societies. This study collected counseling dialogue data via role-playing scenarios involving expert counselors. Third-party counselors evaluated the appropriateness of responses from human counselors and those generated by GPT-4 in identical contexts.
  arXiv  Detail & Related papers  (2024-02-20T06:05:36Z)
• K-ESConv: Knowledge Injection for Emotional Support Dialogue Systems via Prompt Learning [83.19215082550163]
  We propose K-ESConv, a novel prompt learning based knowledge injection method for emotional support dialogue system. We evaluate our model on an emotional support dataset ESConv, where the model retrieves and incorporates knowledge from external professional emotional Q&A forum.
  arXiv  Detail & Related papers  (2023-12-16T08:10:10Z)
• Building Emotional Support Chatbots in the Era of LLMs [64.06811786616471]
  We introduce an innovative methodology that synthesizes human insights with the computational prowess of Large Language Models (LLMs) By utilizing the in-context learning potential of ChatGPT, we generate an ExTensible Emotional Support dialogue dataset, named ExTES. Following this, we deploy advanced tuning techniques on the LLaMA model, examining the impact of diverse training strategies, ultimately yielding an LLM meticulously optimized for emotional support interactions.
  arXiv  Detail & Related papers  (2023-08-17T10:49:18Z)
• Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations [15.230185998553159]
  We develop a novel dataset, named HOPE, to provide a platform for the dialogue-act classification in counselling conversations. We collect 12.9K utterances from publicly-available counselling session videos on YouTube, extract their transcripts, clean, and annotate them with DAC labels. We propose SPARTA, a transformer-based architecture with a novel speaker- and time-aware contextual learning for the dialogue-act classification.
  arXiv  Detail & Related papers  (2021-11-12T10:30:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.