On Large Language Models' Hallucination with Regard to Known Facts

• URL: http://arxiv.org/abs/2403.20009v2
• Date: Mon, 28 Oct 2024 05:57:03 GMT
• Title: On Large Language Models' Hallucination with Regard to Known Facts
• Authors: Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou,
• Abstract summary: Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.
• Score: 74.96789694959894
• License:
• Abstract: Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics, an area not previously covered in studies on hallucinations. We are able to conduct this analysis via two key ideas. First, we identify the factual questions that query the same triplet knowledge but result in different answers. The difference between the model behaviors on the correct and incorrect outputs hence suggests the patterns when hallucinations happen. Second, to measure the pattern, we utilize mappings from the residual streams to vocabulary space. We reveal the different dynamics of the output token probabilities along the depths of layers between the correct and hallucinated cases. In hallucinated cases, the output token's information rarely demonstrates abrupt increases and consistent superiority in the later stages of the model. Leveraging the dynamic curve as a feature, we build a classifier capable of accurately detecting hallucinatory predictions with an 88\% success rate. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.

Related papers

• Distinguishing Ignorance from Error in LLM Hallucinations [43.62904897907926]
  We focus on close-book Question Answering (CBQA), where previous work has not fully addressed the distinction between two possible kinds of hallucinations. We argue that distinguishing these cases is crucial for detecting and mitigating hallucinations.
  arXiv  Detail & Related papers  (2024-10-29T14:31:33Z)
• Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability [83.0884072598828]
  Hallucinations come in many forms, and there is no universally accepted definition. We focus on studying only those hallucinations where a correct answer appears verbatim in the training set. We find that for a fixed dataset, larger and longer-trained LMs hallucinate less. While we see detector size improves performance on fixed LM's outputs, we find an inverse relationship between the scale of the LM and the detectability of its hallucinations.
  arXiv  Detail & Related papers  (2024-08-14T23:34:28Z)
• Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models [65.32990889402927]
  We coin this phenomenon as knowledge overshadowing'' We show that the hallucination rate grows with both the imbalance ratio and the length of dominant condition description. We propose to utilize overshadowing conditions as a signal to catch hallucination before it is produced.
  arXiv  Detail & Related papers  (2024-07-10T20:37:42Z)
• VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models [59.05674402770661]
  This work introduces VideoHallucer, the first comprehensive benchmark for hallucination detection in large video-language models (LVLMs) VideoHallucer categorizes hallucinations into two main types: intrinsic and extrinsic, offering further subcategories for detailed analysis.
  arXiv  Detail & Related papers  (2024-06-24T06:21:59Z)
• In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation [36.31646727970656]
  Large language models (LLMs) frequently hallucinate and produce factual errors. correct generations tend to have sharper context activations in the hidden states of the in-context tokens, compared to the incorrect ones. We propose an entropy-based metric to quantify the sharpness'' among the in-context hidden states and incorporate it into the decoding process.
  arXiv  Detail & Related papers  (2024-03-03T15:53:41Z)
• Alleviating Hallucinations of Large Language Models through Induced Hallucinations [67.35512483340837]
  Large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information. We propose a simple textitInduce-then-Contrast Decoding (ICD) strategy to alleviate hallucinations.
  arXiv  Detail & Related papers  (2023-12-25T12:32:49Z)
• On Early Detection of Hallucinations in Factual Question Answering [4.76359068115052]
  hallucinations remain a major impediment towards gaining user trust. In this work, we explore if the artifacts associated with the model generations can provide hints that the generation will contain hallucinations. Our results show that the distributions of these artifacts tend to differ between hallucinated and non-hallucinated generations.
  arXiv  Detail & Related papers  (2023-12-19T14:35:04Z)
• On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? [32.41234580068662]
  We conduct a study on existing knowledge-grounded conversational benchmarks and several state-of-the-art models. Standard benchmarks consist of >60% hallucinated responses, leading to models that not only hallucinate but even amplify hallucinations. Our findings raise important questions on the quality of existing datasets and models trained using them.
  arXiv  Detail & Related papers  (2022-04-17T05:15:24Z)
• On Hallucination and Predictive Uncertainty in Conditional Language Generation [76.18783678114325]
  Higher predictive uncertainty corresponds to a higher chance of hallucination. Epistemic uncertainty is more indicative of hallucination than aleatoric or total uncertainties. It helps to achieve better results of trading performance in standard metric for less hallucination with the proposed beam search variant.
  arXiv  Detail & Related papers  (2021-03-28T00:32:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.