Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models

• URL: http://arxiv.org/abs/2508.07753v1
• Date: Mon, 11 Aug 2025 08:34:28 GMT
• Title: Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models
• Authors: Zhenliang Zhang, Junzhe Zhang, Xinyu Hu, HuiXuan Zhang, Xiaojun Wan,
• Abstract summary: Large language models (LLMs) have achieved remarkable success in various tasks, yet they remain vulnerable to faithfulness hallucinations.<n>We investigate whether social bias contributes to these hallucinations, a causal relationship that has not been explored.<n>A key challenge is controlling confounders within the context, which complicates the isolation of causality between bias states and hallucinations.
• Score: 50.18087419133284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have achieved remarkable success in various tasks, yet they remain vulnerable to faithfulness hallucinations, where the output does not align with the input. In this study, we investigate whether social bias contributes to these hallucinations, a causal relationship that has not been explored. A key challenge is controlling confounders within the context, which complicates the isolation of causality between bias states and hallucinations. To address this, we utilize the Structural Causal Model (SCM) to establish and validate the causality and design bias interventions to control confounders. In addition, we develop the Bias Intervention Dataset (BID), which includes various social biases, enabling precise measurement of causal effects. Experiments on mainstream LLMs reveal that biases are significant causes of faithfulness hallucinations, and the effect of each bias state differs in direction. We further analyze the scope of these causal effects across various models, specifically focusing on unfairness hallucinations, which are primarily targeted by social bias, revealing the subtle yet significant causal effect of bias on hallucination generation.

Related papers

• A fine-grained look at causal effects in causal spaces [10.99954450966829]
  We study causal effects at the level of events, drawing inspiration from probability theory.<n>We introduce several binary definitions that determine whether a causal effect is present.<n>We show that we can recover the common measures of treatment effect as special cases.
  arXiv  Detail & Related papers  (2025-12-11T14:41:18Z)
• HACK: Hallucinations Along Certainty and Knowledge Axes [66.66625343090743]
  We propose a framework for categorizing hallucinations along two axes: knowledge and certainty.<n>We identify a particularly concerning subset of hallucinations where models hallucinate with certainty despite having the correct knowledge internally.
  arXiv  Detail & Related papers  (2025-10-28T09:34:31Z)
• Do Large Language Models Show Biases in Causal Learning? Insights from Contingency Judgment [0.1547863211792184]
  Causal learning is the cognitive process of developing the capability of making causal inferences.<n>This process is prone to errors and biases, such as the illusion of causality.<n>This cognitive bias has been proposed to underlie many societal problems.
  arXiv  Detail & Related papers  (2025-10-15T18:09:00Z)
• Review of Hallucination Understanding in Large Language and Vision Models [65.29139004945712]
  We present a framework for characterizing both image and text hallucinations across diverse applications.<n>Our investigations reveal that hallucinations often stem from predictable patterns in data distributions and inherited biases.<n>This survey provides a foundation for developing more robust and effective solutions to hallucinations in real-world generative AI systems.
  arXiv  Detail & Related papers  (2025-09-26T09:23:08Z)
• Investigating VLM Hallucination from a Cognitive Psychology Perspective: A First Step Toward Interpretation with Intriguing Observations [60.63340688538124]
  Hallucination is a long-standing problem that has been actively investigated in Vision-Language Models (VLMs)<n>Existing research commonly attributes hallucinations to technical limitations or sycophancy bias, where the latter means the models tend to generate incorrect answers to align with user expectations.<n>In this work, we introduce a psychological taxonomy, categorizing VLMs' cognitive biases that lead to hallucinations, including sycophancy, logical inconsistency, and a newly identified VLMs behaviour: appeal to authority.
  arXiv  Detail & Related papers  (2025-07-03T19:03:16Z)
• Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images [6.48620624181578]
  We introduce SHE (Sequence Hallucination Eradication), a lightweight framework that detects hallucinations and mitigates them.<n>We also propose a new metric (BEACH) to quantify behavioral hallucination severity.
  arXiv  Detail & Related papers  (2025-06-08T15:08:52Z)
• Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models [46.71180299830997]
  Large language models (LLMs) sometimes produce information that appears factually accurate but is, in reality, fabricated.<n>The prevalence of these hallucinations can mislead users, affecting their judgments and decisions.<n>In sectors such as finance, law, and healthcare, such misinformation risks causing substantial economic losses, legal disputes, and health risks.
  arXiv  Detail & Related papers  (2025-06-06T10:50:08Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• Delusions of Large Language Models [62.43923767408462]
  Large Language Models often generate factually incorrect but plausible outputs, known as hallucinations.<n>We identify a more insidious phenomenon, LLM delusion, defined as high belief hallucinations, incorrect outputs with abnormally high confidence, making them harder to detect and mitigate.
  arXiv  Detail & Related papers  (2025-03-09T17:59:16Z)
• Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models [36.327525062842724]
  hallucination is especially concerning in high-stakes domains such as healthcare, legal, and aviation.<n>We examine how factors such as distribution shifts, model size, and model architecture influence hallucination error rate (HER), a metric we introduce to quantify hallucinations.<n>Our findings highlight the importance of incorporating HER alongside traditional metrics like WER to better assess ASR model performance.
  arXiv  Detail & Related papers  (2025-02-18T01:25:39Z)
• Do Large Language Models Show Biases in Causal Learning? [3.0264418764647605]
  Causal learning is the cognitive process of developing the capability of making causal inferences based on available information.<n>This research investigates whether large language models (LLMs) develop causal illusions.
  arXiv  Detail & Related papers  (2024-12-13T19:03:48Z)
• Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models [65.32990889402927]
  We coin this phenomenon as knowledge overshadowing'' We show that the hallucination rate grows with both the imbalance ratio and the length of dominant condition description. We propose to utilize overshadowing conditions as a signal to catch hallucination before it is produced.
  arXiv  Detail & Related papers  (2024-07-10T20:37:42Z)
• On Large Language Models' Hallucination with Regard to Known Facts [74.96789694959894]
  Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.
  arXiv  Detail & Related papers  (2024-03-29T06:48:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.