Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models

• URL: http://arxiv.org/abs/2603.15557v1
• Date: Mon, 16 Mar 2026 17:20:38 GMT
• Title: Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models
• Abstract summary: Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements.<n>We propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition.
• Score: 62.932580559941414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements - posing a critical barrier to their trustworthy deployment. In this work, we propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition. Our framework is grounded in a normative principle of computational rationality, allowing us to model a VLM's generation as a dynamic cognitive trajectory. We design a suite of information-theoretic probes that project this trajectory onto an interpretable, low-dimensional Cognitive State Space. Our central discovery is a governing principle we term the geometric-information duality: a cognitive trajectory's geometric abnormality within this space is fundamentally equivalent to its high information-theoretic surprisal. Hallucination detection is counts as a geometric anomaly detection problem. Evaluated across diverse settings - from rigorous binary QA (POPE) and comprehensive reasoning (MME) to unconstrained open-ended captioning (MS-COCO) - our framework achieves state-of-the-art performance. Crucially, it operates with high efficiency under weak supervision and remains highly robust even when calibration data is heavily contaminated. This approach enables a causal attribution of failures, mapping observable errors to distinct pathological states: perceptual instability (measured by Perceptual Entropy), logical-causal failure (measured by Inferential Conflict), and decisional ambiguity (measured by Decision Entropy). Ultimately, this opens a path toward building AI systems whose reasoning is transparent, auditable, and diagnosable by design.

Related papers

• Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations [27.330265411994258]
  SCHEMA is an evidence-grounded, topology-aware evaluation framework for hallucinations in scientific agents.<n>We show that hallucinations concentrate at a small set of highly connected knowledge hubs.<n>Results indicate that for high-stakes scientific applications, terminal accuracy alone is an insufficient signal of agent reliability.
  arXiv  Detail & Related papers  (2026-08-01T15:19:10Z)
• Beyond the Cartesian Illusion: Testing Two-Stage Multi-Modal Theory of Mind under Perceptual Bottlenecks [16.03812136589077]
  We introduce an Anchor-Based Embodied Spatial Decomposition Chain-of-Thought (CoT)<n>CoT guides the MLLM through a "geometric-to-semantic" projection, forcing it to first establish B's local coordinate system and then dynamically weight visual and auditory modalities based on whether A falls within B's visual frustum.
  arXiv  Detail & Related papers  (2026-05-18T10:32:56Z)
• On Hallucinations in Inverse Problems: Fundamental Limits and Provable Assessment Methods [1.953336755301616]
  Deep neural networks can produce hallucinations, realistic-looking but incorrect details, undermining their reliability.<n>We develop a theoretical framework showing that such hallucinations are not merely artifacts of particular models, but can arise from the ill-posed nature of the inverse problem itself.
  arXiv  Detail & Related papers  (2026-05-13T08:11:43Z)
• Scale-Aware Adversarial Analysis: A Diagnostic for Generative AI in Multiscale Complex Systems [5.173587482427292]
  We introduce a diagnostic framework driven by Constrained Diffusion Decomposition (CDD)<n>We execute deterministic interventions directly within the continuous, CDD-based scale space.<n>We demonstrate that under moderate physical perturbations, the unconstrained generative model exhibits localized structural freezing and non-linear instability.
  arXiv  Detail & Related papers  (2026-05-01T08:36:52Z)
• Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models [65.57985131861399]
  hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states.<n>We propose V-STAR, a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-04-11T13:59:05Z)
• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination [60.197429875410286]
  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems.<n>Formalizing a Neural Uncertainty Principle (NUP) under a loss-induced state, we find that in near-bound regimes, further compression must be accompanied by increased sensitivity dispersion.<n>In vision, masking highly coupled components improves robustness without costly adversarial training.<n>In language, the same prefill-stage probe detects hallucination risk before generating any answer tokens.
  arXiv  Detail & Related papers  (2026-03-20T02:07:10Z)
• PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Multimodal Agents [5.350399160165104]
  multimodal agents evolve from passive observers to long-horizon decision-makers.<n>Current memory systems fail to encode negative constraints.<n>We introduce PolarMem, a training-free Polarized Latent Graph Memory.
  arXiv  Detail & Related papers  (2026-01-31T00:13:56Z)
• Eidoku: A Neuro-Symbolic Verification Gate for LLM Reasoning via Structural Constraint Satisfaction [0.0]
  Large Language Models (LLMs) frequently produce hallucinated statements that are assigned high likelihood by the model itself.<n>This suggests that hallucination is often not a low-confidence phenomenon, but a failure of structural consistency.<n>We reformulate the verification of LLM reasoning as a Constraint Satisfaction Problem (CSP) operating independently of the generation likelihood.
  arXiv  Detail & Related papers  (2025-12-19T05:29:43Z)
• CAVE: Detecting and Explaining Commonsense Anomalies in Visual Environments [34.26006736991816]
  We introduce CAVE, the first benchmark of real-world visual anomalies.<n>CAVE supports three open-ended tasks: anomaly description, explanation, and justification.<n>These annotations draw inspiration from cognitive science research on how humans identify and resolve anomalies.
  arXiv  Detail & Related papers  (2025-10-29T22:34:26Z)
• Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models [4.946483489399819]
  Large Language Models (LLMs) are prone to hallucination, the generation of factually incorrect statements.<n>This work investigates the intrinsic, architectural origins of this failure mode through three primary contributions.
  arXiv  Detail & Related papers  (2025-10-07T16:40:31Z)
• Review of Hallucination Understanding in Large Language and Vision Models [65.29139004945712]
  We present a framework for characterizing both image and text hallucinations across diverse applications.<n>Our investigations reveal that hallucinations often stem from predictable patterns in data distributions and inherited biases.<n>This survey provides a foundation for developing more robust and effective solutions to hallucinations in real-world generative AI systems.
  arXiv  Detail & Related papers  (2025-09-26T09:23:08Z)
• Semantic Energy: Detecting LLM Hallucination Beyond Entropy [106.92072182161712]
  Large Language Models (LLMs) are being increasingly deployed in real-world applications, but they remain susceptible to hallucinations.<n>Uncertainty estimation is a feasible approach to detect such hallucinations.<n>We introduce Semantic Energy, a novel uncertainty estimation framework.
  arXiv  Detail & Related papers  (2025-08-20T07:33:50Z)
• Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models [12.270274049887298]
  Reasoning traces can be redundant or logically inconsistent, making them a new source of hallucination.<n>Existing hallucination detection methods focus primarily on answer-level uncertainty.<n>We propose RACE, a novel framework specifically tailored for hallucination detection in LRMs.
  arXiv  Detail & Related papers  (2025-06-05T09:54:04Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• ACRE: Abstract Causal REasoning Beyond Covariation [90.99059920286484]
  We introduce the Abstract Causal REasoning dataset for systematic evaluation of current vision systems in causal induction. Motivated by the stream of research on causal discovery in Blicket experiments, we query a visual reasoning system with the following four types of questions in either an independent scenario or an interventional scenario. We notice that pure neural models tend towards an associative strategy under their chance-level performance, whereas neuro-symbolic combinations struggle in backward-blocking reasoning.
  arXiv  Detail & Related papers  (2021-03-26T02:42:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.