Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores

• URL: http://arxiv.org/abs/2603.22299v1
• Date: Tue, 17 Mar 2026 08:35:14 GMT
• Title: Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores
• Abstract summary: Large language models (LLMs) are often confidently wrong, making reliable uncertainty estimation (UE) essential.<n>We propose a compact, per-instance UE method that scores cross-layer patterns in internal representations using a single forward pass.
• Score: 36.86032736109853
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models (LLMs) are often confidently wrong, making reliable uncertainty estimation (UE) essential. Output-based heuristics are cheap but brittle, while probing internal representations is effective yet high-dimensional and hard to transfer. We propose a compact, per-instance UE method that scores cross-layer agreement patterns in internal representations using a single forward pass. Across three models, our method matches probing in-distribution, with mean diagonal differences of at most $-1.8$ AUPRC percentage points and $+4.9$ Brier score points. Under cross-dataset transfer, it consistently outperforms probing, achieving off-diagonal gains up to $+2.86$ AUPRC and $+21.02$ Brier points. Under 4-bit weight-only quantization, it remains robust, improving over probing by $+1.94$ AUPRC points and $+5.33$ Brier points on average. Beyond performance, examining specific layer--layer interactions reveals differences in how disparate models encode uncertainty. Altogether, our UE method offers a lightweight, compact means to capture transferable uncertainty in LLMs.

Related papers

• Evaluation Pitfalls and Sparsity Limitations in LLM-based Confidence Estimates for Classification [1.5977080543655726]
  Common approaches such as verbalization produce extremely sparse outputs.<n>Qwen3-32B verbalizes only eight unique confidence values on SST-2, with over half being exactly 95%.<n>We address sparsity and the best AUARC (+2.3 points over vanilla verbalization) without incurring additional inference cost.
  arXiv  Detail & Related papers  (2026-08-05T14:24:34Z)
• Representation Without Reward: A JEPA Audit for LLM Fine-Tuning [1.2691047660244335]
  Joint-embedding predictive architectures (JEPAs) propose that a model should learn more useful abstractions when trained to predict latent representations rather than observed outputs.<n>For autoregressive language-model fine-tuning the principle entails a stricter requirement: the induced hidden-state geometry must reach the language-model head emphand improve the decoded task metric.<n>We test that requirement under a fixed Llama-3.2-1B-Instruct LoRA harness on natural-language-to-regex generation.
  arXiv  Detail & Related papers  (2026-05-14T20:27:32Z)
• When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models [0.0]
  Self-referential statements and meta-cognitive prompts are more stable than paradoxical self-reference on key collapse-related metrics.<n>We propose, as a conjecture, that NCTR forces finite-depth transformers toward dynamical regimes where these problems concentrate.
  arXiv  Detail & Related papers  (2026-04-13T23:23:02Z)
• Spectral Sentinel: Scalable Byzantine-Robust Decentralized Federated Learning via Sketched Random Matrix Theory on Blockchain [0.0]
  Byzantine clients poison gradients under heterogeneous (Non-IID) data.<n>We propose Spectral Sentinel, a Byzantine detection and aggregation framework.<n>We implement the full system with blockchain integration on Polygon networks.
  arXiv  Detail & Related papers  (2025-12-14T09:43:03Z)
• Identity-Link IRT for Label-Free LLM Evaluation: Preserving Additivity in TVD-MI Scores [3.959606869996232]
  We show that averaging TVD-MI's binary trials yields centered-probability scores with additive structure suitable for item-response theory (IRT) without nonlinear link functions.<n>We derive this clipped-linear evaluations from Gini entropy, yielding a box-constrained least-squares formulation that handles boundary saturation.
  arXiv  Detail & Related papers  (2025-10-16T17:59:25Z)
• A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning [40.6234318894435]
  Large language models split into two families: reasoning-centric LLMs and agentic LLMs.<n>This divide arises from fundamentally different training objectives, leading to mismatched strengths and inefficiency on simple queries.<n>We present Adaptive Agent Foundation Model (A$2$FM), a unified framework that follows a route-then-align principle.
  arXiv  Detail & Related papers  (2025-10-13T17:08:25Z)
• CLUE: Non-parametric Verification from Experience via Hidden-State Clustering [64.50919789875233]
  We show that correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations.<n>ClUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates.
  arXiv  Detail & Related papers  (2025-10-02T02:14:33Z)
• Sparsity Forcing: Reinforcing Token Sparsity of MLLMs [40.93786579652003]
  We explicitly reinforce token sparsity in well-posed multimodal large language models (MLLMs) through a simple RL-based post-training framework named textitSparsity Forcing.<n>Our method explores the efficiency-accuracy trade-off by running multiple rollouts with different token budgets, where both efficiency (token reduction ratio) and performance (answer correctness) are formulated as joint rewards.
  arXiv  Detail & Related papers  (2025-04-23T01:45:55Z)
• Locally minimax optimal confidence sets for the best model [40.60301522036206]
  This paper tackles a fundamental inference problem: given $n$ observations from a distribution $P$ over $mathbbRd$ with unknown mean $boldsymbolmu$, we must form a confidence set for the index corresponding to the smallest component of $boldsymbolmu$.<n>We propose "dimension-agnostic" tests that maintain validity regardless of how $d$ scales with $n$, and regardless of arbitrary ties in $boldsymbolmu$.
  arXiv  Detail & Related papers  (2025-03-27T16:06:07Z)
• Uncertainty in Language Models: Assessment through Rank-Calibration [65.10149293133846]
  Language Models (LMs) have shown promising performance in natural language generation. It is crucial to correctly quantify their uncertainty in responding to given inputs. We develop a novel and practical framework, termed $Rank$-$Calibration$, to assess uncertainty and confidence measures for LMs.
  arXiv  Detail & Related papers  (2024-04-04T02:31:05Z)
• Scalable 3D Registration via Truncated Entry-wise Absolute Residuals [65.04922801371363]
  A $3$D registration approach can process more than ten million ($107$) point pairs with over $99%$ random outliers. We call our method TEAR, as it involves minimizing an outlier-robust loss that computes Truncated Entry-wise Absolute Residuals.
  arXiv  Detail & Related papers  (2024-04-01T04:43:39Z)
• Uncertainty-Aware Camera Pose Estimation from Points and Lines [101.03675842534415]
  Perspective-n-Point-and-Line (Pn$PL) aims at fast, accurate and robust camera localizations with respect to a 3D model from 2D-3D feature coordinates.
  arXiv  Detail & Related papers  (2021-07-08T15:19:36Z)
• Certifiably Robust Interpretation via Renyi Differential Privacy [77.04377192920741]
  We study the problem of interpretation robustness from a new perspective of Renyi differential privacy (RDP) First, it can offer provable and certifiable top-$k$ robustness. Second, our proposed method offers $sim10%$ better experimental robustness than existing approaches. Third, our method can provide a smooth tradeoff between robustness and computational efficiency.
  arXiv  Detail & Related papers  (2021-07-04T06:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.