EmpathicStories++: A Multimodal Dataset for Empathy towards Personal Experiences

• URL: http://arxiv.org/abs/2405.15708v1
• Date: Fri, 24 May 2024 16:57:18 GMT
• Title: EmpathicStories++: A Multimodal Dataset for Empathy towards Personal Experiences
• Authors: Jocelyn Shen, Yubin Kim, Mohit Hulse, Wazeer Zulfikar, Sharifa Alghowinem, Cynthia Breazeal, Hae Won Park,
• Abstract summary: EmpathicStories++ is the first longitudinal dataset on empathy, collected over a month-long deployment of social robots in participants' homes. We introduce a novel task of predicting individuals' empathy toward others' stories based on their personal experiences, evaluated in two contexts.
• Score: 19.626851022750067
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling empathy is a complex endeavor that is rooted in interpersonal and experiential dimensions of human interaction, and remains an open problem within AI. Existing empathy datasets fall short in capturing the richness of empathy responses, often being confined to in-lab or acted scenarios, lacking longitudinal data, and missing self-reported labels. We introduce a new multimodal dataset for empathy during personal experience sharing: the EmpathicStories++ dataset (https://mitmedialab.github.io/empathic-stories-multimodal/) containing 53 hours of video, audio, and text data of 41 participants sharing vulnerable experiences and reading empathically resonant stories with an AI agent. EmpathicStories++ is the first longitudinal dataset on empathy, collected over a month-long deployment of social robots in participants' homes, as participants engage in natural, empathic storytelling interactions with AI agents. We then introduce a novel task of predicting individuals' empathy toward others' stories based on their personal experiences, evaluated in two contexts: participants' own personal shared story context and their reflections on stories they read. We benchmark this task using state-of-the-art models to pave the way for future improvements in contextualized and longitudinal empathy modeling. Our work provides a valuable resource for further research in developing empathetic AI systems and understanding the intricacies of human empathy within genuine, real-world settings.

Related papers

• APTNESS: Incorporating Appraisal Theory and Emotion Support Strategies for Empathetic Response Generation [71.26755736617478]
  Empathetic response generation is designed to comprehend the emotions of others. We develop a framework that combines retrieval augmentation and emotional support strategy integration. Our framework can enhance the empathy ability of LLMs from both cognitive and affective empathy perspectives.
  arXiv  Detail & Related papers  (2024-07-23T02:23:37Z)
• Enablers and Barriers of Empathy in Software Developer and User Interaction: A Mixed Methods Case Study [11.260371501613994]
  We studied how empathy is practised between developers and end users. We identified the nature of awareness required to trigger empathy and enablers of empathy. We discovered barriers to empathy and a set of potential strategies to overcome these barriers.
  arXiv  Detail & Related papers  (2024-01-17T06:42:21Z)
• Empathy Detection from Text, Audiovisual, Audio or Physiological Signals: Task Formulations and Machine Learning Methods [5.7306786636466995]
  Detecting empathy has potential applications in society, healthcare and education. Despite being a broad and overlapping topic, the avenue of empathy detection leveraging Machine Learning remains underexplored. We discuss challenges, research gaps and potential applications in the Affective Computing-based empathy domain.
  arXiv  Detail & Related papers  (2023-10-30T08:34:12Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling [59.08591308749448]
  We interviewed eighteen data workers from both industry and academia to learn where and how they would like to collaborate with AI. Surprisingly, though the participants showed excitement about collaborating with AI, many of them also expressed reluctance and pointed out nuanced reasons.
  arXiv  Detail & Related papers  (2023-04-17T15:30:05Z)
• Empathic Conversations: A Multi-level Dataset of Contextualized Conversations [24.54662089036839]
  This dataset is the first to present empathy in multiple forms along with personal distress, emotion, personality characteristics, and person-level demographic information. People differ in their perception of the empathy of others. These differences are associated with certain characteristics such as personality and demographics.
  arXiv  Detail & Related papers  (2022-05-25T11:56:29Z)
• EmpBot: A T5-based Empathetic Chatbot focusing on Sentiments [75.11753644302385]
  Empathetic conversational agents should not only understand what is being discussed, but also acknowledge the implied feelings of the conversation partner. We propose a method based on a transformer pretrained language model (T5) We evaluate our model on the EmpatheticDialogues dataset using both automated metrics and human evaluation.
  arXiv  Detail & Related papers  (2021-10-30T19:04:48Z)
• Modeling User Empathy Elicited by a Robot Storyteller [2.309914459672557]
  We present the first approach to modeling user empathy elicited during interactions with a robotic agent. We conducted experiments with 8 classical machine learning models and 2 deep learning models to detect empathy. Our highest-performing approach, based on XGBoost, achieved an accuracy of 69% and AUC of 72% when detecting empathy in videos.
  arXiv  Detail & Related papers  (2021-07-29T21:56:19Z)
• Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication [88.52901763928045]
  We propose an approach that relies on exemplars to cue the generative model on fine stylistic properties that signal empathy to the interlocutor. We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics.
  arXiv  Detail & Related papers  (2021-06-22T14:02:33Z)
• Towards Persona-Based Empathetic Conversational Models [58.65492299237112]
  Empathetic conversational models have been shown to improve user satisfaction and task outcomes in numerous domains. In Psychology, persona has been shown to be highly correlated to personality, which in turn influences empathy. We propose a new task towards persona-based empathetic conversations and present the first empirical study on the impact of persona on empathetic responding.
  arXiv  Detail & Related papers  (2020-04-26T08:51:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.