Semantic Faithfulness and Entropy Production Measures to Tame Your LLM Demons and Manage Hallucinations

• URL: http://arxiv.org/abs/2512.05156v2
• Date: Mon, 08 Dec 2025 15:12:35 GMT
• Title: Semantic Faithfulness and Entropy Production Measures to Tame Your LLM Demons and Manage Hallucinations
• Abstract summary: We propose two new metrics for faithfulness evaluation using insights from information theory and thermodynamics.<n>We model Question-Context-Answer (QCA) triplets as probability distributions over shared topics.<n>We show that high faithfulness generally implies low entropy production.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Evaluating faithfulness of Large Language Models (LLMs) to a given task is a complex challenge. We propose two new unsupervised metrics for faithfulness evaluation using insights from information theory and thermodynamics. Our approach treats an LLM as a bipartite information engine where hidden layers act as a Maxwell demon controlling transformations of context $C $ into answer $A$ via prompt $Q$. We model Question-Context-Answer (QCA) triplets as probability distributions over shared topics. Topic transformations from $C$ to $Q$ and $A$ are modeled as transition matrices ${\bf Q}$ and ${\bf A}$ encoding the query goal and actual result, respectively. Our semantic faithfulness (SF) metric quantifies faithfulness for any given QCA triplet by the Kullback-Leibler (KL) divergence between these matrices. Both matrices are inferred simultaneously via convex optimization of this KL divergence, and the final SF metric is obtained by mapping the minimal divergence onto the unit interval [0,1], where higher scores indicate greater faithfulness. Furthermore, we propose a thermodynamics-based semantic entropy production (SEP) metric in answer generation, and show that high faithfulness generally implies low entropy production. The SF and SEP metrics can be used jointly or separately for LLM evaluation and hallucination control. We demonstrate our framework on LLM summarization of corporate SEC 10-K filings.

Related papers

• Coherence Under Commitment: Probing Generalization and Vacuous Memorization in LLM Logical Reasoning [0.9023847175654603]
  We introduce Coherence Under Commitment (CUC), a dual-query evaluation paradigm that jointly measures consistency and decisiveness.<n>CUC contributes three innovations: (1) a commitment score $c(varphi) = p(varphi) + p(lnotvarphi)$ quantifying probability mass allocated to decisive outcomes; (2) a textbfdeterministic elicitation protocol via normalized YES/NO log probabilities, eliminating sampling variance; and (3) a 3-way decision framework (True/False/Uncertain) operationalizing the coherence-commitment trade-off into metrics.
  arXiv  Detail & Related papers  (2026-06-19T04:12:56Z)
• Transformers Provably Learn to Internalize Chain-of-Thought [65.41010769606844]
  Implicit Chain-of-Thought (ICoT) trains models to internalize intermediate steps within their hidden states.<n>Log-ICoT removes thinking tokens one at a time, reducing the number of stages from linear in $k$ to logarithmic.<n>Experiments on multi-layer transformers confirm the theory and visualize how reasoning is progressively absorbed into deeper layers.
  arXiv  Detail & Related papers  (2026-05-27T15:17:06Z)
• Representation Without Reward: A JEPA Audit for LLM Fine-Tuning [1.2691047660244335]
  Joint-embedding predictive architectures (JEPAs) propose that a model should learn more useful abstractions when trained to predict latent representations rather than observed outputs.<n>For autoregressive language-model fine-tuning the principle entails a stricter requirement: the induced hidden-state geometry must reach the language-model head emphand improve the decoded task metric.<n>We test that requirement under a fixed Llama-3.2-1B-Instruct LoRA harness on natural-language-to-regex generation.
  arXiv  Detail & Related papers  (2026-05-14T20:27:32Z)
• Collective Alignment in LLM Multi-Agent Systems: Disentangling Bias from Cooperation via Statistical Physics [0.0]
  We present a model-agnostic statistical-physics method to disentangle social conformity from intrinsic bias, compute critical exponents, and probe the collective behavior and possible phase transitions of multi-agent systems.<n>We find that collective alignment is dominated by an intrinsic bias rather than by cooperative neighbor coupling, producing field-driven crossovers instead of genuine phase transitions.
  arXiv  Detail & Related papers  (2026-05-11T13:13:44Z)
• Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models [52.61023005303122]
  We show that low-confidence remasking improves a proxy for quality while constraining the entropy of the induced sequence distribution.<n>We develop a simple Independent--Hastings sampler that approximately targets this distribution during decoding.
  arXiv  Detail & Related papers  (2026-04-01T02:01:30Z)
• $\ abla$-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space [71.23672814629448]
  $nabla$-Reasoner is an iterative generation framework that integrates differentiable optimization over token logits into the decoding loop.<n>$nabla$-Reasoner achieves over 20% accuracy improvement on a challenging mathematical reasoning benchmark.
  arXiv  Detail & Related papers  (2026-03-05T08:42:54Z)
• Adaptation to Intrinsic Dependence in Diffusion Language Models [5.185131234265025]
  Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) approaches.<n>We introduce a distribution-agnostic unmasking schedule for DLMs that adapts to the (unknown) dependence structure of the target data distribution.<n>Our results significantly improve upon prior convergence theories and yield substantial sampling acceleration for low-complexity distributions.
  arXiv  Detail & Related papers  (2026-02-23T18:41:34Z)
• Don't Eliminate Cut: Exponential Separations in LLM-Based Theorem Proving [8.948475969696075]
  We develop a theoretical analysis of LLM-guided formal theorem proving in interactive proof assistants (e.g., Lean)<n>To capture modern representation learning, we treat the state and action spaces as general compact metric spaces and assume Lipschitz policies.<n>Our main separation result shows that when cut elimination expands a DAG of depth $D$ into a cut-free tree of size $(D)$ while the cut-aware hierarchical process has size $O(D)$ with $ll$, a flat learner provably requires exponentially more data than a cut-aware hierarchical learner
  arXiv  Detail & Related papers  (2026-02-11T04:24:09Z)
• Self-Improving World Modelling with Latent Actions [53.93276450137471]
  Internal modelling of the world is essential to reasoning and planning.<n>We propose SWIRL, a self-improvement framework that learns from state-only sequences.
  arXiv  Detail & Related papers  (2026-02-05T19:04:41Z)
• Transformers as Measure-Theoretic Associative Memory: A Statistical Perspective and Minimax Optimality [52.424255020469595]
  Transformers excel through content-addressable retrieval and the ability to exploit contexts, in principle, length.<n>We recast associative memory at the level of probability measures, treating a context as a distribution over unbounded tokens.<n>We show that a shallow measure-theoretic Transformer learns the recall-and-predict map under a spectral assumption on the input densities.
  arXiv  Detail & Related papers  (2026-02-02T09:34:17Z)
• Entropy-Aligned Decoding of LMs for Better Writing and Reasoning [21.971790771470324]
  Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution.<n>Currently, vanilla random sampling from LMs yields low quality generations.<n>We introduce EPIC, a hyper- parameter-free decoding approach that incorporates the entropy of future trajectories into LM decoding.
  arXiv  Detail & Related papers  (2026-01-05T01:37:10Z)
• The 4/$δ$ Bound: Designing Predictable LLM-Verifier Systems for Formal Method Guarantee [5.345468714252351]
  This work bridges the gap by developing an LLM-Verifier Convergence Theorem.<n>We model the interaction between the LLM and the verifier as a discrete-time Markov Chain.<n>We stress-tested this prediction in an extensive empirical campaign comprising more than 90,000 trials.
  arXiv  Detail & Related papers  (2025-11-30T22:19:09Z)
• Reasoning with Confidence: Efficient Verification of LLM Reasoning Steps via Uncertainty Heads [104.9566359759396]
  We propose a lightweight alternative for step-level reasoning verification based on data-driven uncertainty scores.<n>Our findings suggest that the internal states of LLMs encode their uncertainty and can serve as reliable signals for reasoning verification.
  arXiv  Detail & Related papers  (2025-11-09T03:38:29Z)
• A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA [65.38186593873313]
  Multi-Hop Question Answering (MHQA) requires integrating dispersed, interdependent evidence through sequential reasoning under noise.<n>We introduce a proof-of-concept multi-call framework for MHQA, InfoQA.<n>We construct a stringent and noise-rich benchmark to validate our theory and framework.
  arXiv  Detail & Related papers  (2025-09-25T14:11:57Z)
• Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity [15.16188621701658]
  Hallucination in large language models can be detected by assessing the uncertainty of model outputs, typically measured using entropy.<n>We propose a simple black-box uncertainty quantification method inspired by nearest neighbor estimates of entropy.<n>Our approach can also be easily extended to white-box settings by incorporating token probabilities.
  arXiv  Detail & Related papers  (2025-05-30T21:21:05Z)
• HypRL: Reinforcement Learning of Control Policies for Hyperproperties [0.0]
  We propose HYPRL, a specification-guided reinforcement learning framework.<n>We apply Skolemization to manage quantifier alternations and define quantitative functions to shape rewards.<n>A suitable RL algorithm is then used to learn policies that collectively maximize the expected reward.
  arXiv  Detail & Related papers  (2025-04-07T01:58:36Z)
• Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking [41.3496135369579]
  Chain-of-thought (CoT) significantly enhances the performance of large language models (LLMs) across a wide range of tasks.<n>There is limited mechanistic understanding of the algorithms that Transformer+CoT can learn.<n>We evaluate the state tracking capabilities of Transformer+CoT and its variants, confirming the effectiveness of CoT.
  arXiv  Detail & Related papers  (2025-02-27T14:24:51Z)
• FLARE: Faithful Logic-Aided Reasoning and Exploration [47.46564769245296]
  We introduce a novel approach for traversing the problem space using task decompositions.<n>We use the Large Language Models to plan a solution, soft-formalise the query into facts and predicates using a logic programming code.<n>Our method allows us to compute the faithfulness of the reasoning process w.r.t. the generated code and analyse the steps of the multi-hop search without relying on external solvers.
  arXiv  Detail & Related papers  (2024-10-14T19:39:11Z)
• Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers [54.20763128054692]
  We study how a two-attention-layer transformer is trained to perform ICL on $n$-gram Markov chain data. We prove that the gradient flow with respect to a cross-entropy ICL loss converges to a limiting model.
  arXiv  Detail & Related papers  (2024-09-09T18:10:26Z)
• Quantifying Semantic Emergence in Language Models [31.608080868988825]
  Large language models (LLMs) are widely recognized for their exceptional capacity to capture semantics meaning.<n>In this work, we introduce a quantitative metric, Information Emergence (IE), designed to measure LLMs' ability to extract semantics from input tokens.
  arXiv  Detail & Related papers  (2024-05-21T09:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.