Detecting LLM Hallucinations via Embedding Cluster Geometry: A Three-Type Taxonomy with Measurable Signatures

• URL: http://arxiv.org/abs/2602.14259v1
• Date: Sun, 15 Feb 2026 18:14:10 GMT
• Title: Detecting LLM Hallucinations via Embedding Cluster Geometry: A Three-Type Taxonomy with Measurable Signatures
• Abstract summary: We propose a geometric taxonomy of large language model hallucinations based on observable signatures in token embedding cluster structure.<n>We identify three operationally distinct hallucination types: Type 1 (center-drift) under weak context, Type 2 (wrong-well convergence) to locally coherent but contextually incorrect cluster regions, and Type 3 (coverage gaps) where no cluster structure exists.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a geometric taxonomy of large language model hallucinations based on observable signatures in token embedding cluster structure. By analyzing the static embedding spaces of 11 transformer models spanning encoder (BERT, RoBERTa, ELECTRA, DeBERTa, ALBERT, MiniLM, DistilBERT) and decoder (GPT-2) architectures, we identify three operationally distinct hallucination types: Type 1 (center-drift) under weak context, Type 2 (wrong-well convergence) to locally coherent but contextually incorrect cluster regions, and Type 3 (coverage gaps) where no cluster structure exists. We introduce three measurable geometric statistics: α (polarity coupling), \b{eta} (cluster cohesion), and λ_s (radial information gradient). Across all 11 models, polarity structure (α > 0.5) is universal (11/11), cluster cohesion (\b{eta} > 0) is universal (11/11), and the radial information gradient is significant (9/11, p < 0.05). We demonstrate that the two models failing λ_s significance -- ALBERT and MiniLM -- do so for architecturally explicable reasons: factorized embedding compression and distillation-induced isotropy, respectively. These findings establish the geometric prerequisites for type-specific hallucination detection and yield testable predictions about architecture-dependent vulnerability profiles.

Related papers

• Hyper-Fold: Exploring the Expressive Limit of Sequence-Geometry Learning for Proteins via Hypergraph Modeling [55.36837305276515]
  We show that the complete bilinear operator over content-geometry outer products--the sufficient statistic of all second-order interactions--is the expressive ceiling.<n>We then introduce Hyper-Fold, a rank-K separable convolutional backbone approaching this ceiling at message-passing cost.
  arXiv  Detail & Related papers  (2026-08-29T11:43:25Z)
• The Hallucination Signal Is a Mean Shift: Why Simple Probes Suffice [24.66711854680014]
  Hidden-state probes effectively detect LLM hallucinations, but the geometry of the signal remains poorly characterized.<n>Across three 7B-scale models and three datasets in a paired-example paradigm, we find the signal overwhelmingly dominated by a single mean-shift component.<n> Shrinkage linear discriminant analysis closes about 73% of the gap between 1D and full-dimensional classifiers.<n>Because the signal spans a contiguous layer band, LayerMix aggregates it to match oracle-layer performance without oracle access.
  arXiv  Detail & Related papers  (2026-08-28T23:01:21Z)
• Mapping and Measuring the Behavioral Evolution of Large Language Models [28.603443859513153]
  Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across generations.<n>We characterize the output behavior of 32 models from six families using their responses to a shared bank of 10,000 prompts.
  arXiv  Detail & Related papers  (2026-08-11T15:07:28Z)
• Finding Belief Geometries with Sparse Autoencoders [0.0]
  We introduce a pipeline for discovering candidate simplex-structured subspaces in transformer representations.<n>We validate the pipeline on a transformer trained on a hidden Markov model with known belief-state geometry.<n>Preliminary evidence that genuine belief-like geometry exists in Gemma-2-9B's representation space.
  arXiv  Detail & Related papers  (2026-04-03T03:29:48Z)
• Whitening Reveals Cluster Commitment as the Geometric Separator of Hallucination Types [0.0]
  A geometric taxonomy distinguishes three failure types -- center-drift (Type1), wrong-well convergence (Type2), and coverage gaps (Type3)<n>Prior work found Types1 and2 indistinguishable in full-dimensional contextual measurement.<n>We address this through PCA-whitening and eigenspectrum decomposition on GPT-2-small.
  arXiv  Detail & Related papers  (2026-03-08T18:12:10Z)
• A Geometric Taxonomy of Hallucinations in LLMs [0.2538209532048866]
  The term "hallucination" in large language models conflates distinct phenomena with different geometric signatures in embedding space.<n>We propose a taxonomy identifying three types: unfaithfulness, confabulation, and factual error.<n>The contribution is a geometric taxonomy clarifying the scope of embedding-based detection.
  arXiv  Detail & Related papers  (2026-01-26T22:07:09Z)
• Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection [2.8547732086436306]
  A fundamental limitation of supervised deep learning is "Generalization Collapse"<n>We propose Latent Sculpting, a hierarchical two-stage representation learning framework.<n>We report an 88.89% detection rate on "Infiltration" scenarios.
  arXiv  Detail & Related papers  (2025-12-19T11:37:02Z)
• Aligned Manifold Property and Topology Point Clouds for Learning Molecular Properties [55.2480439325792]
  This work introduces AMPTCR, a molecular surface representation that combines local quantum-derived scalar fields and custom topological descriptors within an aligned point cloud format.<n>For molecular weight, results confirm that AMPTCR encodes physically meaningful data, with a validation R2 of 0.87.<n>In the bacterial inhibition task, AMPTCR enables both classification and direct regression of E. coli inhibition values.
  arXiv  Detail & Related papers  (2025-07-22T04:35:50Z)
• How high is `high'? Rethinking the roles of dimensionality in topological data analysis and manifold learning [8.397730500554047]
  We present a generalised Hanson-Wright inequality and use it to establish new statistical insights into the geometry of data point-clouds.<n>We revisit the ground-breaking neuroscience discovery of isometric toroidal structure in grid-cell activity made by Gardner et al.<n>Our findings reveal, for the first time, evidence that this structure is in fact to physical space, meaning that grid cell activity conveys a geometrically faithful representation of the real world.
  arXiv  Detail & Related papers  (2025-05-22T16:34:15Z)
• (Deep) Generative Geodesics [57.635187092922976]
  We introduce a newian metric to assess the similarity between any two data points. Our metric leads to the conceptual definition of generative distances and generative geodesics. Their approximations are proven to converge to their true values under mild conditions.
  arXiv  Detail & Related papers  (2024-07-15T21:14:02Z)
• Curved Geometric Networks for Visual Anomaly Recognition [39.91252195360767]
  Learning a latent embedding to understand the underlying nature of data distribution is often formulated in Euclidean spaces with zero curvature. In this work, we investigate benefits of the curved space for analyzing anomalies or out-of-distribution objects in data.
  arXiv  Detail & Related papers  (2022-08-02T01:15:39Z)
• GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning [55.79997930181418]
  Generalized Zero-Shot Learning aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. We propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation.
  arXiv  Detail & Related papers  (2022-07-05T04:04:37Z)
• Linear Connectivity Reveals Generalization Strategies [54.947772002394736]
  Some pairs of finetuned models have large barriers of increasing loss on the linear paths between them. We find distinct clusters of models which are linearly connected on the test loss surface, but are disconnected from models outside the cluster. Our work demonstrates how the geometry of the loss surface can guide models towards different functions.
  arXiv  Detail & Related papers  (2022-05-24T23:43:02Z)
• Multiway Spherical Clustering via Degree-Corrected Tensor Block Models [8.147652597876862]
  We develop a degree-corrected block model with estimation accuracy guarantees. In particular, we demonstrate that an intrinsic statistical-to-computational gap emerges only for tensors of order three or greater. The efficacy of our procedure is demonstrated through two data applications.
  arXiv  Detail & Related papers  (2022-01-19T03:40:22Z)
• Mitigating Generation Shifts for Generalized Zero-Shot Learning [52.98182124310114]
  Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. We propose a novel Generation Shifts Mitigating Flow framework for learning unseen data synthesis efficiently and effectively. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings.
  arXiv  Detail & Related papers  (2021-07-07T11:43:59Z)
• Quantum anomalous Hall phase in synthetic bilayers via twistless twistronics [58.720142291102135]
  We propose quantum simulators of "twistronic-like" physics based on ultracold atoms and syntheticdimensions. We show that our system exhibits topologicalband structures under appropriate conditions.
  arXiv  Detail & Related papers  (2020-08-06T19:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.