Large Language Models Produce Responses Perceived to be Empathic

• URL: http://arxiv.org/abs/2403.18148v1
• Date: Tue, 26 Mar 2024 23:14:34 GMT
• Title: Large Language Models Produce Responses Perceived to be Empathic
• Authors: Yoon Kyung Lee, Jina Suh, Hongli Zhan, Junyi Jessy Li, Desmond C. Ong,
• Abstract summary: Large Language Models (LLMs) generate empathic messages in response to posts describing common life experiences. We showed human raters a variety of responses written by several models, and had people rate these responses on how empathic they seemed to be. We found that LLM-generated responses were consistently rated as more empathic than human-written responses.
• Score: 40.38391275905264
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have demonstrated surprising performance on many tasks, including writing supportive messages that display empathy. Here, we had these models generate empathic messages in response to posts describing common life experiences, such as workplace situations, parenting, relationships, and other anxiety- and anger-eliciting situations. Across two studies (N=192, 202), we showed human raters a variety of responses written by several models (GPT4 Turbo, Llama2, and Mistral), and had people rate these responses on how empathic they seemed to be. We found that LLM-generated responses were consistently rated as more empathic than human-written responses. Linguistic analyses also show that these models write in distinct, predictable ``styles", in terms of their use of punctuation, emojis, and certain words. These results highlight the potential of using LLMs to enhance human peer support in contexts where empathy is important.

Related papers

• Can Machines Resonate with Humans? Evaluating the Emotional and Empathic Comprehension of LMs [31.556095945149583]
  We propose several strategies to improve empathy understanding in language models. A low agreement among annotators hinders training and highlights the subjective nature of the task. To study this, we meticulously collected story pairs in Urdu language and find that subjectivity in interpreting empathy among annotators appears to be independent of cultural background.
  arXiv  Detail & Related papers  (2024-06-17T06:22:20Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations [19.11368665202549]
  When large language models are applied in the field of psychological counseling, they often rush to provide universal advice. We constructed a multi-turn empathetic conversation dataset of more than 2 million samples. Experiments have shown that the empathy ability of LLMs can be significantly enhanced when finetuning by using multi-turn dialogue history.
  arXiv  Detail & Related papers  (2023-11-01T03:49:52Z)
• Harnessing Large Language Models' Empathetic Response Generation Capabilities for Online Mental Health Counselling Support [1.9336815376402723]
  Large Language Models (LLMs) have demonstrated remarkable performance across various information-seeking and reasoning tasks. This study sought to examine LLMs' capability to generate empathetic responses in conversations that emulate those in a mental health counselling setting. We selected five LLMs: version 3.5 and version 4 of the Generative Pre-training (GPT), Vicuna FastChat-T5, Pathways Language Model (PaLM) version 2, and Falcon-7B-Instruct.
  arXiv  Detail & Related papers  (2023-10-12T03:33:06Z)
• Editing Personality for Large Language Models [73.59001811199823]
  This paper introduces an innovative task focused on editing the personality traits of Large Language Models (LLMs) We construct PersonalityEdit, a new benchmark dataset to address this task.
  arXiv  Detail & Related papers  (2023-10-03T16:02:36Z)
• Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBench [83.41621219298489]
  We evaluate Large Language Models' (LLMs) anthropomorphic capabilities using the emotion appraisal theory from psychology. We collect a dataset containing over 400 situations that have proven effective in eliciting the eight emotions central to our study. We conduct a human evaluation involving more than 1,200 subjects worldwide.
  arXiv  Detail & Related papers  (2023-08-07T15:18:30Z)
• EmpHi: Generating Empathetic Responses with Human-like Intents [15.209172627484246]
  We propose a novel model to generate empathetic responses with human-consistent empathetic intents, EmpHi. EmHi learns the distribution of potential empathetic intents with a discrete latent variable, then combines both implicit and explicit intent representation to generate responses with various empathetic intents. Experiments show that EmpHi outperforms state-of-the-art models in terms of empathy, relevance, and diversity on both automatic and human evaluation.
  arXiv  Detail & Related papers  (2022-04-26T09:49:49Z)
• Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication [88.52901763928045]
  We propose an approach that relies on exemplars to cue the generative model on fine stylistic properties that signal empathy to the interlocutor. We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics.
  arXiv  Detail & Related papers  (2021-06-22T14:02:33Z)
• Towards Persona-Based Empathetic Conversational Models [58.65492299237112]
  Empathetic conversational models have been shown to improve user satisfaction and task outcomes in numerous domains. In Psychology, persona has been shown to be highly correlated to personality, which in turn influences empathy. We propose a new task towards persona-based empathetic conversations and present the first empirical study on the impact of persona on empathetic responding.
  arXiv  Detail & Related papers  (2020-04-26T08:51:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.