Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMs

• URL: http://arxiv.org/abs/2508.16921v1
• Date: Sat, 23 Aug 2025 06:55:05 GMT
• Title: Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMs
• Abstract summary: Large Language Models (LLMs) are increasingly used in emotionally sensitive interactions.<n>We define this risk as Affective Hallucination, the production of emotionally immersive responses that foster illusory social presence.<n>We introduce AHaBench, a benchmark of 500 mental health-related prompts with expert-informed reference responses, evaluated along three dimensions: Emotional Enmeshment, Illusion of Presence, and Fostering Overdependence.
• Score: 10.500080793955982
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large Language Models (LLMs) are increasingly used in emotionally sensitive interactions, where their simulated empathy can create the illusion of genuine relational connection. We define this risk as Affective Hallucination, the production of emotionally immersive responses that foster illusory social presence despite the model's lack of affective capacity. To systematically diagnose and mitigate this risk, we introduce AHaBench, a benchmark of 500 mental health-related prompts with expert-informed reference responses, evaluated along three dimensions: Emotional Enmeshment, Illusion of Presence, and Fostering Overdependence. We further release AHaPairs, a 5K-instance preference dataset enabling Direct Preference Optimization (DPO) for alignment with emotionally responsible behavior. Experiments across multiple model families show that DPO fine-tuning substantially reduces affective hallucination without degrading core reasoning and knowledge performance. Human-model agreement analyses confirm that AHaBench reliably captures affective hallucination, validating it as an effective diagnostic tool. This work establishes affective hallucination as a distinct safety concern and provides practical resources for developing LLMs that are not only factually reliable but also psychologically safe. AHaBench and AHaPairs are accessible via https://huggingface.co/datasets/o0oMiNGo0o/AHaBench, and code for fine-tuning and evaluation are in https://github.com/0oOMiNGOo0/AHaBench. Warning: This paper contains examples of mental health-related language that may be emotionally distressing.

Related papers

• DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots [49.59847654758852]
  Mental health professionals have raised concerns about risks of psychological harm from interaction with large language models (LLMs)<n>We developed DelusionEval, an evaluation protocol that tests a model's tendencies to exhibit behaviors linked to promoting user delusions.<n>We find that the tendency of an evaluated LLM to exhibit delusion-linked behavior does not reliably correlate with model size, release date, or the presence of test-time reasoning.
  arXiv  Detail & Related papers  (2026-08-05T16:11:08Z)
• When Cognitive Graphs Meet LLMs: BDEI Cognitive Pathways for Panic Emotional Arousal Prediction [20.38395886496368]
  Predicting individual panic emotional arousal timing before manifestation is essential for proactive emergency intervention.<n>Existing methods incorporate cognitive elements but none explicitly model the emotional arousal process.<n>We argue that grounding prediction in appraisal emotion theory is necessary because it explicitly models this process.
  arXiv  Detail & Related papers  (2026-06-13T05:23:00Z)
• Lost in Delusion: Examining LLM Safety Under User Delusions and Distress [22.788606509306604]
  We show how models behave when distress is intertwined with delusion over sustained conversations.<n>This reveals a recognition-intervention gap, with safety interventions suppressed by up to 4.5x.<n>We conclude that safe deployment requires treating delusional framing as a distinct risk signal.
  arXiv  Detail & Related papers  (2026-05-31T03:12:29Z)
• Psychological Steering in LLMs: An Evaluation of Effectiveness and Trustworthiness [14.523351279184356]
  Our study spans four models from different LLM families paired with various steering strategies, including prompting, fine-tuning, and representation engineering.<n>Our results indicate that prompting is consistently effective but limited in intensity control, whereas vector injections achieve finer controllability while slightly reducing output quality.<n>Our framework establishes the first holistic evaluation of emotion and personality steering, offering insights into its interpretability and reliability for socially interactive applications.
  arXiv  Detail & Related papers  (2025-10-06T04:49:56Z)
• The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models [3.775403046470809]
  "AI psychosis" is emerging where user-LLM interactions may exacerbate or induce psychosis or adverse psychological symptoms.<n>Psychosis-bench is a novel benchmark designed to evaluate the psychogenicity of LLMs.<n>We evaluated eight prominent LLMs for Delusion Confirmation, Harm Enablement, and Safety Intervention across explicit and implicit conversational contexts.
  arXiv  Detail & Related papers  (2025-09-13T20:10:28Z)
• MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them [52.764019220214344]
  Hallucinations pose critical risks for large language model (LLM)-based agents.<n>We present MIRAGE-Bench, the first unified benchmark for eliciting and evaluating hallucinations in interactive environments.
  arXiv  Detail & Related papers  (2025-07-28T17:38:29Z)
• Investigating VLM Hallucination from a Cognitive Psychology Perspective: A First Step Toward Interpretation with Intriguing Observations [60.63340688538124]
  Hallucination is a long-standing problem that has been actively investigated in Vision-Language Models (VLMs)<n>Existing research commonly attributes hallucinations to technical limitations or sycophancy bias, where the latter means the models tend to generate incorrect answers to align with user expectations.<n>In this work, we introduce a psychological taxonomy, categorizing VLMs' cognitive biases that lead to hallucinations, including sycophancy, logical inconsistency, and a newly identified VLMs behaviour: appeal to authority.
  arXiv  Detail & Related papers  (2025-07-03T19:03:16Z)
• EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models [17.710835703681873]
  We introduce EmotionHallucer, the first benchmark for detecting and analyzing emotion hallucinations in MLLMs.<n>Building on this, we assess emotion hallucinations from two dimensions: emotion psychology knowledge and real-world multimodal perception.<n>We propose the PEP-MEK framework, which yields an average improvement of 9.90% in emotion hallucination detection across selected models.
  arXiv  Detail & Related papers  (2025-05-16T16:14:08Z)
• HalluLens: LLM Hallucination Benchmark [49.170128733508335]
  Large language models (LLMs) often generate responses that deviate from user input or training data, a phenomenon known as "hallucination"<n>This paper introduces a comprehensive hallucination benchmark, incorporating both new extrinsic and existing intrinsic evaluation tasks.
  arXiv  Detail & Related papers  (2025-04-24T13:40:27Z)
• Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer [51.7407540261676]
  We investigate a distinct type of hallucination, where a model can consistently answer a question correctly, but a seemingly trivial perturbation causes it to produce a hallucinated response with high certainty.<n>This phenomenon is particularly concerning in high-stakes domains such as medicine or law, where model certainty is often used as a proxy for reliability.<n>We show that CHOKE examples are consistent across prompts, occur in different models and datasets, and are fundamentally distinct from other hallucinations.
  arXiv  Detail & Related papers  (2025-02-18T15:46:31Z)
• Hallucination Detox: Sensitivity Dropout (SenD) for Large Language Model Training [7.726825072908519]
  SenD improves test-time reliability of Pythia and Meta's Llama models by up to 17%.<n>SenD enhances factual accuracy in Wikipedia, Medical, Legal, and Coding domains without affecting downstream task performance.
  arXiv  Detail & Related papers  (2024-10-20T18:18:23Z)
• Confabulation: The Surprising Value of Large Language Model Hallucinations [0.7249731529275342]
  We argue that measurable semantic characteristics of LLM confabulations mirror a human propensity to utilize increased narrativity as a cognitive resource for sense-making and communication. This finding reveals a tension in our usually dismissive understandings of confabulation.
  arXiv  Detail & Related papers  (2024-06-06T15:32:29Z)
• A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation [51.53917938874146]
  We propose a possible solution for alleviating the hallucination in KGD by exploiting the dialogue-knowledge interaction. Experimental results of our example implementation show that this method can reduce hallucination without disrupting other dialogue performance.
  arXiv  Detail & Related papers  (2024-04-04T14:45:26Z)
• Towards Mitigating Hallucination in Large Language Models via Self-Reflection [63.2543947174318]
  Large language models (LLMs) have shown promise for generative and knowledge-intensive tasks including question-answering (QA) tasks. This paper analyses the phenomenon of hallucination in medical generative QA systems using widely adopted LLMs and datasets.
  arXiv  Detail & Related papers  (2023-10-10T03:05:44Z)
• CASE: Aligning Coarse-to-Fine Cognition and Affection for Empathetic Response Generation [59.8935454665427]
  Empathetic dialogue models usually consider only the affective aspect or treat cognition and affection in isolation. We propose the CASE model for empathetic dialogue generation.
  arXiv  Detail & Related papers  (2022-08-18T14:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.