Thermodynamic Signatures of Reasoning: Free-Energy and Spectral-Form-Factor Diagnostics for Hallucination Detection in Large Language Models

• URL: http://arxiv.org/abs/2606.19404v1
• Date: Wed, 17 Jun 2026 11:42:48 GMT
• Title: Thermodynamic Signatures of Reasoning: Free-Energy and Spectral-Form-Factor Diagnostics for Hallucination Detection in Large Language Models
• Abstract summary: Hallucination detection in large language models (LLMs) is deployment-critical, and recent work shows that the spectrum of attention-derived graph Laplacians carries strong signal about reasoning quality.<n>We propose Free-Energy Signatures (Fes), a spectral descriptor that treats each layer's attention Laplacian as a Hamiltonian and extracts its thermodynamic potentials partition function, free energy, spectral entropy, heat capacity together with the random-matrix-theory (RMT) spectral form factor.
• Score: 1.0152838128195467
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hallucination detection in large language models (LLMs) is deployment-critical, and recent work shows that the spectrum of attention-derived graph Laplacians carries strong signal about reasoning quality. Prior spectral diagnostics, however, summarize the Laplacian spectrum by a handful of eigenvalues or hand-picked scalars, leaving most of its structure unused. We propose Free-Energy Signatures (Fes), a spectral descriptor that treats each layer's attention Laplacian as a Hamiltonian and extracts its thermodynamic potentials partition function, free energy, spectral entropy, heat capacity together with the random-matrix-theory (RMT) spectral form factor. We prove three results: (i)~Lipschitz stability of Fes under attention perturbation; (ii)~an expressiveness result showing that Fes enriches finite spectral summaries and approximates moment-derived spectral functionals under explicit regularity and grid-resolution assumptions; and (iii)~a finite-sample PAC bound on the AUROC of a training-free detector built from Fes. Empirically, across six open-weight LLMs and six benchmarks, a lightweight probe on Fes descriptors achieves the strongest aggregate AUROC among attention-spectral baselines, improving over LapEig by $+6.5$ AUROC points and over GoR-4 by $+2.4$ points on average, while requiring no update to the underlying LLM. In the fully unsupervised setting, an RMT-deviation score achieves mean AUROC $0.71$, providing a label-free but weaker detector. A complementary RMT analysis shows that correct generations exhibit more Wigner-Dyson like spectral statistics, whereas hallucinations exhibit more Poisson-like statistics. The anonymized code and config are provided in the supplementary material.

Related papers

• Revisiting Spectral Representations in Generative Diffusion Models [98.43552218868837]
  Diffusion models have shown remarkable performance on diverse generation tasks.<n>Recent work finds that imposing representation alignment on the hidden states of diffusion networks can both facilitate training convergence and enhance sampling quality.<n>We investigate the connection between self-supervised spectral representation learning and diffusion generative models through a shared perspective on perturbation kernels.
  arXiv  Detail & Related papers  (2026-09-08T04:54:02Z)
• S3C-LLM: Skill-Code Guided Agentic Language Models for Spectrum-to-Structure Elucidation [12.882727231856279]
  We introduce S3C-LLM, a skill-guided and code-grounded agentic LLM for spectrum-to-structure elucidation.<n>Rather than directly predicting a molecule, S3C-LLM retrieves modality-specific spectroscopy skills, executes analysis code to instantiate these skills on the input spectra, and integrates the resulting peak-level evidence and formula constraints before generating SMILES.
  arXiv  Detail & Related papers  (2026-08-31T14:55:32Z)
• Fully Rotation-Equivariant Spectral-Spatial Learning for Multispectral Object Detection [26.632380150006753]
  FressDet is a fully rotation-equivariant spectral-spatial learning framework for multispectral object detection.<n>It captures the continuous, ordered nature of spectral structure and enables reliable spectral-spatial fusion across pyramid levels.<n>FressDet learns more discriminative and robust spectral-spatial representations even under rotational perturbations.
  arXiv  Detail & Related papers  (2026-07-06T14:29:51Z)
• Spectral Model eXplainer: a chemically-grounded explainability framework for spectral-based machine learning models [0.6487259764989486]
  This study introduces the Spectral Model eXplainer (SMX), a post-hoc, global, model-agnostic XAI framework that explains spectral classifiers through expert-informed spectral zones.<n>SMX summarizes each zone via PCA, defines quantile-based logical predicates, estimates relevance with perturbations, and aggregates bag-wise rankings in a directed weighted graph.
  arXiv  Detail & Related papers  (2026-05-04T15:01:04Z)
• XANE(3): An E(3)-Equivariant Graph Neural Network for Accurate Prediction of XANES Spectra from Atomic Structures [5.177624503826816]
  XANE(3) is a physics-based E(3)-equivariant graph neural network for predicting X-ray absorption near-edge structure (XANES) spectra directly from atomic structures.<n>We evaluate the model on a dataset of 5,941 FDMNES simulations of iron oxide surface facets and obtain a spectrum mean squared error of $1.0 times 10-3$ on the test set.
  arXiv  Detail & Related papers  (2026-04-13T23:42:14Z)
• Robust Graph Representation Learning via Adaptive Spectral Contrast [0.9558392439655014]
  High-frequency signals are indispensable for encoding heterophily but exhibit significantly higher variance under spectrally concentrated perturbations.<n>We propose ASPECT, a framework that resolves this dilemma through a reliability-aware spectral gating mechanism.<n>ASPECT achieves new state-of-the-art performance on 8 out of 9 benchmarks, effectively decoupling meaningful structural heterophily from incidental noise.
  arXiv  Detail & Related papers  (2026-04-02T10:38:00Z)
• On the Spectral Flattening of Quantized Embeddings [25.64641307046705]
  Training Large Language Models at ultra-low precision is critically impeded by instability rooted in the conflict between discrete quantization constraints and the intrinsic heavy-tailed spectral nature of linguistic data.<n>This work not only quantifies the spectral sensitivity of LLMs but also establishes spectral fidelity as a necessary condition for stable low-bit optimization.
  arXiv  Detail & Related papers  (2026-02-01T02:21:53Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Suppressing VLM Hallucinations with Spectral Representation Filtering [49.52703800684483]
  Vision-language models (VLMs) frequently produce hallucinations in the form of descriptions of objects, attributes, or relations that do not exist in the image.<n>We introduce Spectral Representation Filtering (SRF), a lightweight, training-free method to suppress such hallucinations by analyzing and correcting the covariance structure of the model's representations.
  arXiv  Detail & Related papers  (2025-11-15T13:49:27Z)
• SpectrumFM: Redefining Spectrum Cognition via Foundation Modeling [65.65474629224558]
  We propose a spectrum foundation model, termed SpectrumFM, which provides a new paradigm for spectrum cognition.<n>An innovative spectrum encoder that exploits the convolutional neural networks is proposed to effectively capture both fine-grained local signal structures and high-level global dependencies in the spectrum data.<n>Two novel self-supervised learning tasks, namely masked reconstruction and next-slot signal prediction, are developed for pre-training SpectrumFM, enabling the model to learn rich and transferable representations.
  arXiv  Detail & Related papers  (2025-08-02T14:40:50Z)
• Hallucination Detection in LLMs with Topological Divergence on Attention Graphs [60.83579255387347]
  Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models.<n>We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting.
  arXiv  Detail & Related papers  (2025-04-14T10:06:27Z)
• Towards Ultimate NMR Resolution with Deep Learning [0.0]
  In multidimensional NMR spectroscopy, practical resolution is defined as the ability to distinguish and accurately determine signal positions against a background of overlapping peaks, thermal noise, and spectral artifacts.<n>In the pursuit of ultimate resolution, we introduce Peak Probability presentations ($P3$)- a statistical spectral representation that assigns a probability to each spectral point, indicating the likelihood of a peak maximum occurring at that location.<n>The mapping between the spectrum and $P3$ is achieved using MR-Ai, a physics-inspired deep learning neural network architecture, designed to handle multidimensional NMR spectra.
  arXiv  Detail & Related papers  (2025-02-28T07:20:25Z)
• Unsupervised Spectral Unmixing For Telluric Correction Using A Neural Network Autoencoder [58.720142291102135]
  We present a neural network autoencoder approach for extracting a telluric transmission spectrum from a large set of high-precision observed solar spectra from the HARPS-N radial velocity spectrograph.
  arXiv  Detail & Related papers  (2021-11-17T12:54:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.