SENSE-7: Taxonomy and Dataset for Measuring User Perceptions of Empathy in Sustained Human-AI Conversations

• URL: http://arxiv.org/abs/2509.16437v1
• Date: Fri, 19 Sep 2025 21:32:24 GMT
• Title: SENSE-7: Taxonomy and Dataset for Measuring User Perceptions of Empathy in Sustained Human-AI Conversations
• Authors: Jina Suh, Lindy Le, Erfan Shayegani, Gonzalo Ramos, Judith Amores, Desmond C. Ong, Mary Czerwinski, Javier Hernandez,
• Abstract summary: We propose a human-centered taxonomy that emphasizes observable empathic behaviors.<n>We introduce a new dataset, Sense-7, of real-world conversations between information workers and Large Language Models (LLMs)<n>Analysis of 695 conversations from 109 participants reveals that empathy judgments are highly individualized, context-sensitive, and vulnerable to disruption.
• Score: 13.232694774856931
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Empathy is increasingly recognized as a key factor in human-AI communication, yet conventional approaches to "digital empathy" often focus on simulating internal, human-like emotional states while overlooking the inherently subjective, contextual, and relational facets of empathy as perceived by users. In this work, we propose a human-centered taxonomy that emphasizes observable empathic behaviors and introduce a new dataset, Sense-7, of real-world conversations between information workers and Large Language Models (LLMs), which includes per-turn empathy annotations directly from the users, along with user characteristics, and contextual details, offering a more user-grounded representation of empathy. Analysis of 695 conversations from 109 participants reveals that empathy judgments are highly individualized, context-sensitive, and vulnerable to disruption when conversational continuity fails or user expectations go unmet. To promote further research, we provide a subset of 672 anonymized conversation and provide exploratory classification analysis, showing that an LLM-based classifier can recognize 5 levels of empathy with an encouraging average Spearman $\rho$=0.369 and Accuracy=0.487 over this set. Overall, our findings underscore the need for AI designs that dynamically tailor empathic behaviors to user contexts and goals, offering a roadmap for future research and practical development of socially attuned, human-centered artificial agents.

Related papers

• Reflecting Twice before Speaking with Empathy: Self-Reflective Alternating Inference for Empathy-Aware End-to-End Spoken Dialogue [53.95386201009769]
  We introduce EmpathyEval, a descriptive natural-language-based evaluation model for assessing empathetic quality in spoken dialogues.<n>We propose ReEmpathy, an end-to-end Spoken Language Models that enhances empathetic dialogue through a novel Empathetic Self-Reflective Alternating Inference mechanism.
  arXiv  Detail & Related papers  (2026-01-26T09:04:50Z)
• HumanLLM: Towards Personalized Understanding and Simulation of Human Nature [72.55730315685837]
  HumanLLM is a foundation model designed for personalized understanding and simulation of individuals.<n>We first construct the Cognitive Genome, a large-scale corpus curated from real-world user data on platforms like Reddit, Twitter, Blogger, and Amazon.<n>We then formulate diverse learning tasks and perform supervised fine-tuning to empower the model to predict a wide range of individualized human behaviors, thoughts, and experiences.
  arXiv  Detail & Related papers  (2026-01-22T09:27:27Z)
• From Measurement to Expertise: Empathetic Expert Adapters for Context-Based Empathy in Conversational AI Agents [9.297250791971837]
  We introduce a novel framework for developing and evaluating context-specific empathetic large language models (LLMs)<n>We analyze a real-world conversational dataset consisting of 672 multi-turn conversations across 8 tasks.<n>We train empathetic expert adapters for context-specific empathy that specialize in varying empathy levels based on the recognized task.
  arXiv  Detail & Related papers  (2025-11-05T03:07:27Z)
• E-THER: A Multimodal Dataset for Empathic AI - Towards Emotional Mismatch Awareness [3.8298581733964903]
  E-THER is the first Person-Centered Therapy-grounded multimodal dataset with multidimensional annotations for verbal-visual incongruence detection.<n>We show that our incongruence-trained models outperform general-purpose models in critical traits.
  arXiv  Detail & Related papers  (2025-09-02T08:58:32Z)
• Bridging Cognition and Emotion: Empathy-Driven Multimodal Misinformation Detection [56.644686934050576]
  Social media has become a major conduit for information dissemination, yet it also facilitates the rapid spread of misinformation.<n>Traditional misinformation detection methods primarily focus on surface-level features, overlooking the crucial roles of human empathy in the propagation process.<n>We propose the Dual-Aspect Empathy Framework (DAE), which integrates cognitive and emotional empathy to analyze misinformation from both the creator and reader perspectives.
  arXiv  Detail & Related papers  (2025-04-24T07:48:26Z)
• Multi-dimensional Evaluation of Empathetic Dialog Responses [4.580983642743026]
  We propose a multi-dimensional empathy evaluation framework to measure both emphexpressed intents from the speaker's perspective and emphperceived empathy from the listener's perspective. We find the two dimensions are inter-connected, while perceived empathy has high correlations with dialogue satisfaction levels.
  arXiv  Detail & Related papers  (2024-02-18T00:32:33Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning [29.262204241732565]
  Existing methods assume that all emotions-of-interest are given a priori as annotated training examples. We conceptualize one-shot recognition of emotions in context -- a new problem aimed at recognizing human affect states in finer particle level from a single support sample. All variants of our model clearly outperform the random baseline, while leveraging the semantic scene context consistently improves the learnt representations.
  arXiv  Detail & Related papers  (2021-11-30T10:35:20Z)
• EmpBot: A T5-based Empathetic Chatbot focusing on Sentiments [75.11753644302385]
  Empathetic conversational agents should not only understand what is being discussed, but also acknowledge the implied feelings of the conversation partner. We propose a method based on a transformer pretrained language model (T5) We evaluate our model on the EmpatheticDialogues dataset using both automated metrics and human evaluation.
  arXiv  Detail & Related papers  (2021-10-30T19:04:48Z)
• CEM: Commonsense-aware Empathetic Response Generation [31.956147246779423]
  We propose a novel approach for empathetic response generation, which leverages commonsense to draw more information about the user's situation. We evaluate our approach on EmpatheticDialogues, which is a widely-used benchmark dataset for empathetic response generation.
  arXiv  Detail & Related papers  (2021-09-13T06:55:14Z)
• Exemplars-guided Empathetic Response Generation Controlled by the Elements of Human Communication [88.52901763928045]
  We propose an approach that relies on exemplars to cue the generative model on fine stylistic properties that signal empathy to the interlocutor. We empirically show that these approaches yield significant improvements in empathetic response quality in terms of both automated and human-evaluated metrics.
  arXiv  Detail & Related papers  (2021-06-22T14:02:33Z)
• Towards Persona-Based Empathetic Conversational Models [58.65492299237112]
  Empathetic conversational models have been shown to improve user satisfaction and task outcomes in numerous domains. In Psychology, persona has been shown to be highly correlated to personality, which in turn influences empathy. We propose a new task towards persona-based empathetic conversations and present the first empirical study on the impact of persona on empathetic responding.
  arXiv  Detail & Related papers  (2020-04-26T08:51:01Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.