First Hallucination Tokens Are Different from Conditional Ones

• URL: http://arxiv.org/abs/2507.20836v1
• Date: Mon, 28 Jul 2025 13:44:21 GMT
• Title: First Hallucination Tokens Are Different from Conditional Ones
• Abstract summary: Hallucinations at the token level are vital for real-time filtering and targeted correction.<n>We leverage the RAGTruth corpus with token-level annotations and reproduced logits to analyse hallucination signals.<n>Our results show that the first hallucinated token carries a stronger signal and is more detectable than conditional tokens.
• Score: 17.13749013546228
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hallucination, the generation of untruthful content, is one of the major concerns regarding foundational models. Detecting hallucinations at the token level is vital for real-time filtering and targeted correction, yet the variation of hallucination signals within token sequences is not fully understood. Leveraging the RAGTruth corpus with token-level annotations and reproduced logits, we analyse how these signals depend on a token's position within hallucinated spans, contributing to an improved understanding of token-level hallucination. Our results show that the first hallucinated token carries a stronger signal and is more detectable than conditional tokens. We release our analysis framework, along with code for logit reproduction and metric computation at https://github.com/jakobsnl/RAGTruth_Xtended.

Related papers

• UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations [61.85916280854257]
  Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input.<n>We introduce textbfUniProbe, a lightweight, unified, learnable detector that models a frozen LVLM's heterogeneous computational trace from a single forward pass.
  arXiv  Detail & Related papers  (2026-08-11T12:01:59Z)
• Scalable Token-Level Hallucination Detection in Large Language Models [63.3426544914783]
  Internal hallucinations are difficult to detect in reasoning-intensive tasks.<n>We propose TokenHD, a holistic pipeline for training token-level hallucination detectors.
  arXiv  Detail & Related papers  (2026-05-12T16:47:40Z)
• Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding [8.629393350514972]
  Token-Guard is a token-level hallucination control method based on self-checking decoding.<n> Experiments on HALU datasets show Token-Guard substantially reduces hallucinations and improves generation accuracy.
  arXiv  Detail & Related papers  (2026-01-29T16:48:47Z)
• Measuring the Impact of Lexical Training Data Coverage on Hallucination Detection in Large Language Models [26.89705770151822]
  Hallucination in large language models (LLMs) is a fundamental challenge, particularly in open-domain question answering.<n>Prior work attempts to detect hallucination with model-internal signals such as token-level entropy or generation consistency.<n>We investigate whether data coverage itself can serve as a detection signal.
  arXiv  Detail & Related papers  (2025-11-22T06:59:55Z)
• Image Tokens Matter: Mitigating Hallucination in Discrete Tokenizer-based Large Vision-Language Models via Latent Editing [39.969451863788464]
  Large Vision-Language Models (LVLMs) unify multimodal representations by encoding visual inputs into a finite set of tokens.<n>We find that these models still hallucinate non-existent objects.<n>We propose a hallucination mitigation method that suppresses the influence of visually absent tokens by modifying latent image embeddings during generation.
  arXiv  Detail & Related papers  (2025-05-24T22:36:15Z)
• Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding [33.33247964758369]
  We argue that adequate contextual information can be extracted directly from the token interaction process.<n>Inspired by causal inference in the decoding strategy, we propose to leverage causal masks to establish information propagation between multimodal tokens.<n>We present FarSight, a versatile plug-and-play decoding strategy to reduce attention interference from outlier tokens.
  arXiv  Detail & Related papers  (2025-05-22T13:19:57Z)
• Mitigating Hallucinations in Vision-Language Models through Image-Guided Head Suppression [6.838584336878126]
  Large vision language models (LVLMs) often suffer from hallucinations, generating texts misaligned with the visual context.<n>Existing methods aimed at reducing hallucinations through inference time intervention incur a significant increase in latency.<n>We present SPIN, a task-agnostic attention-guided head suppression strategy that can be seamlessly integrated during inference.
  arXiv  Detail & Related papers  (2025-05-22T09:00:57Z)
• Diverging Towards Hallucination: Detection of Failures in Vision-Language Models via Multi-token Aggregation [1.9226023650048942]
  Vision-language models (VLMs) now rival human performance on many multimodal tasks, yet they still hallucinate objects or generate unsafe text.<n>We demonstrate that analyzing the complete sequence of early logits potentially provides substantially more diagnostic information.<n>We introduce Multi-Token Reliability Estimation (MTRE), a lightweight, white-box method that aggregates logits from the first ten tokens using multi-token log-likelihood ratios and self-attention.
  arXiv  Detail & Related papers  (2025-05-16T23:00:19Z)
• HalluLens: LLM Hallucination Benchmark [49.170128733508335]
  Large language models (LLMs) often generate responses that deviate from user input or training data, a phenomenon known as "hallucination"<n>This paper introduces a comprehensive hallucination benchmark, incorporating both new extrinsic and existing intrinsic evaluation tasks.
  arXiv  Detail & Related papers  (2025-04-24T13:40:27Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection [3.1269598124014264]
  We introduce ANHALTEN, a dataset that extends the English hallucination detection dataset to German. This is the first work that explores cross-lingual transfer for token-level reference-free hallucination detection. We show that the sample-efficient few-shot transfer is the most effective approach in most setups.
  arXiv  Detail & Related papers  (2024-07-18T17:01:38Z)
• ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models [65.12177400764506]
  Large language models (LLMs) exhibit hallucinations in long-form question-answering tasks across various domains and wide applications.<n>Current hallucination detection and mitigation datasets are limited in domains and sizes.<n>This paper introduces an iterative self-training framework that simultaneously and progressively scales up the hallucination annotation dataset.
  arXiv  Detail & Related papers  (2024-07-05T17:56:38Z)
• Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment [52.43197107069751]
  Multimodal Large Language Models (MLLMs) often generate factually inaccurate information, referred to as hallucination.<n>We introduce Data-augmented Phrase-level Alignment (DPA), a novel loss which can be applied to instruction-tuned off-the-shelf MLLMs to mitigate hallucinations.
  arXiv  Detail & Related papers  (2024-05-28T23:36:00Z)
• On Large Language Models' Hallucination with Regard to Known Facts [74.96789694959894]
  Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.
  arXiv  Detail & Related papers  (2024-03-29T06:48:30Z)
• ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models [6.014286500397164]
  Hallucinations in vision-language models pose a significant challenge to their reliability, particularly in the generation of long captions. We introduce ESREAL, a novel unsupervised learning framework designed to suppress the generation of hallucinations through accurate localization and penalization of hallucinated tokens. Our framework notably reduces hallucinations in LLaVA, InstructBLIP, and mPLUG-Owl2 by 32.81%, 27.08%, and 7.46% on the CHAIR metric.
  arXiv  Detail & Related papers  (2024-03-24T14:21:06Z)
• Alleviating Hallucinations of Large Language Models through Induced Hallucinations [67.35512483340837]
  Large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information. We propose a simple textitInduce-then-Contrast Decoding (ICD) strategy to alleviate hallucinations.
  arXiv  Detail & Related papers  (2023-12-25T12:32:49Z)
• On Early Detection of Hallucinations in Factual Question Answering [4.76359068115052]
  hallucinations remain a major impediment towards gaining user trust. In this work, we explore if the artifacts associated with the model generations can provide hints that the generation will contain hallucinations. Our results show that the distributions of these artifacts tend to differ between hallucinated and non-hallucinated generations.
  arXiv  Detail & Related papers  (2023-12-19T14:35:04Z)
• OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation [124.9008419182485]
  We present OPERA, a novel MLLM decoding method grounded in an Over-trust Penalty and a Retrospection-Allocation strategy. Our approach begins with an interesting observation that, most hallucinations are closely tied to the knowledge aggregation patterns in the self-attention matrix. Based on the observation, OPERA introduces a penalty term on the model logits during the beam-search decoding to mitigate the over-trust issue.
  arXiv  Detail & Related papers  (2023-11-29T18:57:07Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)
• Mutual Information Alleviates Hallucinations in Abstractive Summarization [73.48162198041884]
  We find a simple criterion under which models are significantly more likely to assign more probability to hallucinated content during generation: high model uncertainty. This finding offers a potential explanation for hallucinations: models default to favoring text with high marginal probability, when uncertain about a continuation. We propose a decoding strategy that switches to optimizing for pointwise mutual information of the source and target token--rather than purely the probability of the target token--when the model exhibits uncertainty.
  arXiv  Detail & Related papers  (2022-10-24T13:30:54Z)
• Detecting Hallucinated Content in Conditional Neural Sequence Generation [165.68948078624499]
  We propose a task to predict whether each token in the output sequence is hallucinated (not contained in the input) We also introduce a method for learning to detect hallucinations using pretrained language models fine tuned on synthetic data.
  arXiv  Detail & Related papers  (2020-11-05T00:18:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.