SymLoc: Symbolic Localization of Hallucination across HaluEval and TruthfulQA

• URL: http://arxiv.org/abs/2511.14172v1
• Date: Tue, 18 Nov 2025 06:16:58 GMT
• Title: SymLoc: Symbolic Localization of Hallucination across HaluEval and TruthfulQA
• Abstract summary: LLMs still struggle with hallucination, especially when confronted with symbolic triggers like modifier, negation, numbers, exceptions, and named entities.<n>We propose the first symbolic localization framework to meaningfully trace the development of hallucinations across all model layers.
• Score: 9.274826797278582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLMs still struggle with hallucination, especially when confronted with symbolic triggers like modifiers, negation, numbers, exceptions, and named entities. Yet, we lack a clear understanding of where these symbolic hallucinations originate, making it crucial to systematically handle such triggers and localize the emergence of hallucination inside the model. While prior work explored localization using statistical techniques like LSC and activation variance analysis, these methods treat all tokens equally and overlook the role symbolic linguistic knowledge plays in triggering hallucinations. So far, no approach has investigated how symbolic elements specifically drive hallucination failures across model layers, nor has symbolic linguistic knowledge been used as the foundation for a localization framework. We propose the first symbolic localization framework that leverages symbolic linguistic and semantic knowledge to meaningfully trace the development of hallucinations across all model layers. By focusing on how models process symbolic triggers, we analyze five models using HaluEval and TruthfulQA. Our symbolic knowledge approach reveals that attention variance for these linguistic elements explodes to critical instability in early layers (2-4), with negation triggering catastrophic variance levels, demonstrating that symbolic semantic processing breaks down from the very beginning. Through the lens of symbolic linguistic knowledge, despite larger model sizes, hallucination rates remain consistently high (78.3%-83.7% across Gemma variants), with steep attention drops for symbolic semantic triggers throughout deeper layers. Our findings demonstrate that hallucination is fundamentally a symbolic linguistic processing failure, not a general generation problem, revealing that symbolic semantic knowledge provides the key to understanding and localizing hallucination mechanisms in LLMs.

Related papers

• HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models [70.67112531913669]
  Hallucinations in vision language models are commonly treated as semantic errors.<n>Prior work mainly focuses on detecting or suppressing hallucinations at generation time.<n>We study Post Hallucination Reasoning (PHR), the stage in which hallucinated semantics enter the model's inference context.
  arXiv  Detail & Related papers  (2026-07-08T15:02:11Z)
• Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments [19.486391934670028]
  Large Language Models (LLMs) are prone to factual hallucinations, risking their reliability in real-world applications.<n>Existing hallucination detectors mainly extract micro-level intrinsic patterns for uncertainty quantification or elicit macro-level self-judgments through verbalized prompts.<n>We propose LaaB, a framework that bridges neural features and symbolic judgments for hallucination detection.
  arXiv  Detail & Related papers  (2026-05-05T16:53:20Z)
• Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation [50.51650162235191]
  We propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention.<n>IVE is effective across various base MLLMs and multiple hallucination benchmarks, particularly for cognitive hallucinations.
  arXiv  Detail & Related papers  (2026-04-02T12:51:07Z)
• Bridging Day and Night: Target-Class Hallucination Suppression in Unpaired Image Translation [37.38870850999494]
  Day-to-night unpaired image translation is challenging due to large appearance shifts and lack of direct pixel-level supervision.<n>Existing methods often introduce semantic hallucinations, where objects from target classes such as traffic signs and vehicles, as well as man-made light effects, are incorrectly synthesized.<n>We propose a novel framework that detects and suppresses hallucinations of target-class features during unpaired translation.
  arXiv  Detail & Related papers  (2026-02-17T06:37:00Z)
• SHALE: A Scalable Benchmark for Fine-grained Hallucination Evaluation in LVLMs [52.03164192840023]
  Large Vision-Language Models (LVLMs) still suffer from hallucinations, i.e., generating content inconsistent with input or established world knowledge.<n>We propose an automated data construction pipeline that produces scalable, controllable, and diverse evaluation data.<n>We construct SHALE, a benchmark designed to assess both faithfulness and factuality hallucinations.
  arXiv  Detail & Related papers  (2025-08-13T07:58:01Z)
• When Semantics Mislead Vision: Mitigating Large Multimodal Models Hallucinations in Scene Text Spotting and Understanding [75.57997630182136]
  We investigate the underlying causes of semantic hallucination and identify a key finding: Transformer layers in Large Multimodal Models with stronger attention focus on scene text regions are less prone to producing semantic hallucinations.<n>We propose a training-free semantic hallucination mitigation framework comprising two key components: ZoomText and Grounded Layer Correction.<n>Our method not only effectively mitigates semantic hallucination but also achieves strong performance on public benchmarks for scene text spotting and understanding.
  arXiv  Detail & Related papers  (2025-06-05T19:53:19Z)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM [58.2298313720146]
  Multimodal hallucinations are multi-sourced and arise from diverse causes.<n>Existing benchmarks fail to adequately distinguish between perception-induced hallucinations and reasoning-induced hallucinations.
  arXiv  Detail & Related papers  (2025-05-30T05:54:36Z)
• HalluLens: LLM Hallucination Benchmark [49.170128733508335]
  Large language models (LLMs) often generate responses that deviate from user input or training data, a phenomenon known as "hallucination"<n>This paper introduces a comprehensive hallucination benchmark, incorporating both new extrinsic and existing intrinsic evaluation tasks.
  arXiv  Detail & Related papers  (2025-04-24T13:40:27Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• Simulation of Non-Ordinary Consciousness [0.0]
  generative symbolic interface designed to simulate psilocybin-like symbolic cognition.<n>Glyph consistently generates high-entropy, metaphor-saturated, and ego-dissolving language.
  arXiv  Detail & Related papers  (2025-03-29T23:04:04Z)
• Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens [7.806633929976787]
  Hallucinations in Large Vision-Language Models (LVLMs) significantly undermine their reliability.<n>This paper addresses how LVLMs process visual information and whether this process causes hallucination.<n>We propose a simple inference-time method that adjusts visual attention by integrating information across various heads.
  arXiv  Detail & Related papers  (2024-11-23T03:40:05Z)
• ESREAL: Exploiting Semantic Reconstruction to Mitigate Hallucinations in Vision-Language Models [6.014286500397164]
  Hallucinations in vision-language models pose a significant challenge to their reliability, particularly in the generation of long captions. We introduce ESREAL, a novel unsupervised learning framework designed to suppress the generation of hallucinations through accurate localization and penalization of hallucinated tokens. Our framework notably reduces hallucinations in LLaVA, InstructBLIP, and mPLUG-Owl2 by 32.81%, 27.08%, and 7.46% on the CHAIR metric.
  arXiv  Detail & Related papers  (2024-03-24T14:21:06Z)
• Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models [52.957842999317506]
  Object hallucination refers to the phenomenon that the LVLMs claim non-existent objects in the image. We propose a Logical Closed Loop-based framework for Object Hallucination Detection and Mitigation, namely LogicCheckGPT. As a plug-and-play method, it can be seamlessly applied to all existing LVLMs.
  arXiv  Detail & Related papers  (2024-02-18T15:28:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.