EMP-EVAL: A Framework for Measuring Empathy in Open Domain Dialogues

• URL: http://arxiv.org/abs/2301.12510v1
• Date: Sun, 29 Jan 2023 18:42:19 GMT
• Title: EMP-EVAL: A Framework for Measuring Empathy in Open Domain Dialogues
• Authors: Bushra Amjad, Muhammad Zeeshan and Mirza Omer Beg
• Abstract summary: EMP-EVAL is a simple yet effective automatic empathy evaluation method. The proposed technique takes the influence of Emotion, Cognitive and Emotional empathy. We show that our metrics can correlate with human preference, achieving comparable results with human judgments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Measuring empathy in conversation can be challenging, as empathy is a complex and multifaceted psychological construct that involves both cognitive and emotional components. Human evaluations can be subjective, leading to inconsistent results. Therefore, there is a need for an automatic method for measuring empathy that reduces the need for human evaluations. In this paper, we proposed a novel approach EMP-EVAL, a simple yet effective automatic empathy evaluation method. The proposed technique takes the influence of Emotion, Cognitive and Emotional empathy. To the best knowledge, our work is the first to systematically measure empathy without the human-annotated provided scores. Experimental results demonstrate that our metrics can correlate with human preference, achieving comparable results with human judgments.

Related papers

• APTNESS: Incorporating Appraisal Theory and Emotion Support Strategies for Empathetic Response Generation [71.26755736617478]
  Empathetic response generation is designed to comprehend the emotions of others. We develop a framework that combines retrieval augmentation and emotional support strategy integration. Our framework can enhance the empathy ability of LLMs from both cognitive and affective empathy perspectives.
  arXiv  Detail & Related papers  (2024-07-23T02:23:37Z)
• Enablers and Barriers of Empathy in Software Developer and User Interaction: A Mixed Methods Case Study [11.260371501613994]
  We studied how empathy is practised between developers and end users. We identified the nature of awareness required to trigger empathy and enablers of empathy. We discovered barriers to empathy and a set of potential strategies to overcome these barriers.
  arXiv  Detail & Related papers  (2024-01-17T06:42:21Z)
• Wish I Can Feel What You Feel: A Neural Approach for Empathetic Response Generation [2.5255184843886225]
  We propose a novel approach, which integrates the three components - emotion cause, knowledge graph, and communication mechanism for empathetic response generation. Experimental results show that incorporating the key components generates more informative and empathetic responses.
  arXiv  Detail & Related papers  (2022-12-05T03:20:37Z)
• CASE: Aligning Coarse-to-Fine Cognition and Affection for Empathetic Response Generation [59.8935454665427]
  Empathetic dialogue models usually consider only the affective aspect or treat cognition and affection in isolation. We propose the CASE model for empathetic dialogue generation.
  arXiv  Detail & Related papers  (2022-08-18T14:28:38Z)
• EmpBot: A T5-based Empathetic Chatbot focusing on Sentiments [75.11753644302385]
  Empathetic conversational agents should not only understand what is being discussed, but also acknowledge the implied feelings of the conversation partner. We propose a method based on a transformer pretrained language model (T5) We evaluate our model on the EmpatheticDialogues dataset using both automated metrics and human evaluation.
  arXiv  Detail & Related papers  (2021-10-30T19:04:48Z)
• Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication [88.52901763928045]
  We propose an approach that relies on exemplars to cue the generative model on fine stylistic properties that signal empathy to the interlocutor. We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics.
  arXiv  Detail & Related papers  (2021-06-22T14:02:33Z)
• Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction [55.47134146639492]
  This article proposes a unifed end-to-end neural architecture, which is capable of simultaneously encoding the semantics and the emotions in a post. Experiments on real-world data demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both content coherence and emotion appropriateness.
  arXiv  Detail & Related papers  (2021-06-06T06:26:15Z)
• Towards an Online Empathetic Chatbot with Emotion Causes [10.700455393948818]
  It is critical to learn the causes that evoke the users' emotion for empathetic responding. To gather emotion causes in online environments, we leverage counseling strategies. We verify the effectiveness of the proposed approach by comparing our judgements with several SOTA methods.
  arXiv  Detail & Related papers  (2021-05-11T02:52:46Z)
• MIME: MIMicking Emotions for Empathetic Response Generation [82.57304533143756]
  Current approaches to empathetic response generation view the set of emotions expressed in the input text as a flat structure. We argue that empathetic responses often mimic the emotion of the user to a varying degree, depending on its positivity or negativity and content.
  arXiv  Detail & Related papers  (2020-10-04T00:35:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.