Breakdowns in Conversational AI: Interactional Failures in Emotionally and Ethically Sensitive Contexts

• URL: http://arxiv.org/abs/2604.02713v1
• Date: Fri, 03 Apr 2026 04:10:46 GMT
• Title: Breakdowns in Conversational AI: Interactional Failures in Emotionally and Ethically Sensitive Contexts
• Abstract summary: We develop a persona-conditioned user simulator capable of engaging in multi-turn dialogue with psychological personas and staged emotional pacing.<n>We identify several common failure patterns, including affective misalignments, ethical failures, and cross-dimensional trade-offs where empathy supersedes responsibility.
• Score: 16.938835302814084
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversational AI is increasingly deployed in emotionally charged and ethically sensitive interactions. Previous research has primarily concentrated on emotional benchmarks or static safety checks, overlooking how alignment unfolds in evolving conversation. We explore the research question: what breakdowns arise when conversational agents confront emotionally and ethically sensitive behaviors, and how do these affect dialogue quality? To stress-test chatbot performance, we develop a persona-conditioned user simulator capable of engaging in multi-turn dialogue with psychological personas and staged emotional pacing. Our analysis reveals that mainstream models exhibit recurrent breakdowns that intensify as emotional trajectories escalate. We identify several common failure patterns, including affective misalignments, ethical guidance failures, and cross-dimensional trade-offs where empathy supersedes or undermines responsibility. We organize these patterns into a taxonomy and discuss the design implications, highlighting the necessity to maintain ethical coherence and affective sensitivity throughout dynamic interactions. The study offers the HCI community a new perspective on the diagnosis and improvement of conversational AI in value-sensitive and emotionally charged contexts.

Related papers

• TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech [15.290622671304305]
  We introduce DyadEE, a dataset for emotional entrainment detection in dyadic speech interactions.<n>We propose TRACE, a window-level framework that models dyadic interaction as ordered sequences of acoustic embeddings derived from emotion fine-tuned Whisper representations.
  arXiv  Detail & Related papers  (2026-06-29T16:42:24Z)
• EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Spoken Dialogue Systems [25.920610612373608]
  EMO-Reasoning is a benchmark for assessing emotional coherence in dialogue systems.<n>We propose the Cross-turn Emotion Reasoning Score to assess the emotion transitions in multi-turn dialogues.
  arXiv  Detail & Related papers  (2025-08-25T03:01:40Z)
• Are You Listening to Me? Fine-Tuning Chatbots for Empathetic Dialogue [0.5849783371898033]
  We explore how Large Language Models (LLMs) respond when tasked with generating emotionally rich interactions.<n>We analyzed the emotional progression of the dialogues using both sentiment analysis (via VADER) and expert assessments.
  arXiv  Detail & Related papers  (2025-07-03T11:32:41Z)
• CauESC: A Causal Aware Model for Emotional Support Conversation [79.4451588204647]
  Existing approaches ignore the emotion causes of the distress. They focus on the seeker's own mental state rather than the emotional dynamics during interaction between speakers. We propose a novel framework CauESC, which firstly recognizes the emotion causes of the distress, as well as the emotion effects triggered by the causes.
  arXiv  Detail & Related papers  (2024-01-31T11:30:24Z)
• Dynamic Causal Disentanglement Model for Dialogue Emotion Detection [77.96255121683011]
  We propose a Dynamic Causal Disentanglement Model based on hidden variable separation. This model effectively decomposes the content of dialogues and investigates the temporal accumulation of emotions. Specifically, we propose a dynamic temporal disentanglement model to infer the propagation of utterances and hidden variables.
  arXiv  Detail & Related papers  (2023-09-13T12:58:09Z)
• Empathetic Dialogue Generation via Sensitive Emotion Recognition and Sensible Knowledge Selection [47.60224978460442]
  We propose a Serial and Emotion-Knowledge interaction (SEEK) method for empathetic dialogue generation. We use a fine-grained encoding strategy which is more sensitive to the emotion dynamics (emotion flow) in the conversations to predict the emotion-intent characteristic of response. Besides, we design a novel framework to model the interaction between knowledge and emotion to generate more sensible response.
  arXiv  Detail & Related papers  (2022-10-21T03:51:18Z)
• Emotion-aware Chat Machine: Automatic Emotional Response Generation for Human-like Emotional Interaction [55.47134146639492]
  This article proposes a unifed end-to-end neural architecture, which is capable of simultaneously encoding the semantics and the emotions in a post. Experiments on real-world data demonstrate that the proposed method outperforms the state-of-the-art methods in terms of both content coherence and emotion appropriateness.
  arXiv  Detail & Related papers  (2021-06-06T06:26:15Z)
• Disambiguating Affective Stimulus Associations for Robot Perception and Dialogue [67.89143112645556]
  We provide a NICO robot with the ability to learn the associations between a perceived auditory stimulus and an emotional expression. NICO is able to do this for both individual subjects and specific stimuli, with the aid of an emotion-driven dialogue system. The robot is then able to use this information to determine a subject's enjoyment of perceived auditory stimuli in a real HRI scenario.
  arXiv  Detail & Related papers  (2021-03-05T20:55:48Z)
• Knowledge Bridging for Empathetic Dialogue Generation [52.39868458154947]
  Lack of external knowledge makes empathetic dialogue systems difficult to perceive implicit emotions and learn emotional interactions from limited dialogue history. We propose to leverage external knowledge, including commonsense knowledge and emotional lexical knowledge, to explicitly understand and express emotions in empathetic dialogue generation.
  arXiv  Detail & Related papers  (2020-09-21T09:21:52Z)
• Generating Emotionally Aligned Responses in Dialogues using Affect Control Theory [15.848210524718219]
  Affect Control Theory (ACT) is a socio-mathematical model of emotions for human-human interactions. We investigate how ACT can be used to develop affect-aware neural conversational agents.
  arXiv  Detail & Related papers  (2020-03-07T19:31:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.