The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models

• URL: http://arxiv.org/abs/2602.08159v1
• Date: Sun, 08 Feb 2026 23:27:10 GMT
• Title: The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models
• Abstract summary: We characterize correctness representations across 9 models from 5 architecture families.<n>We validate causally through activation steering.<n> correctness signal exists internally but is not expressed in outputs.
• Score: 1.45309944076563
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When a language model asserts that "the capital of Australia is Sydney," does it know this is wrong? We characterize the geometry of correctness representations across 9 models from 5 architecture families. The structure is simple: the discriminative signal occupies 3-8 dimensions, performance degrades with additional dimensions, and no nonlinear classifier improves over linear separation. Centroid distance in the low-dimensional subspace matches trained probe performance (0.90 AUC), enabling few-shot detection: on GPT-2, 25 labeled examples achieve 89% of full-data accuracy. We validate causally through activation steering: the learned direction produces 10.9 percentage point changes in error rates while random directions show no effect. Internal probes achieve 0.80-0.97 AUC; output-based methods (P(True), semantic entropy) achieve only 0.44-0.64 AUC. The correctness signal exists internally but is not expressed in outputs. That centroid distance matches probe performance indicates class separation is a mean shift, making detection geometric rather than learned.

Related papers

• GLID: Gated Local Intrinsic Dimension Repairs the Blind Spots of Face-Forgery Detectors [4.564848621539382]
  Fine-tuned foundation-model detectors dominate face-forgery benchmarks, yet they stay blind to generator families absent from training.<n>We present GLID, a detector that repairs this blind spot with geometry instead of data.<n>GLID treats the patch tokens of a single image as a sample from a manifold and estimates their local intrinsic dimension (LID) at several depths of a frozen vision transformer.
  arXiv  Detail & Related papers  (2026-07-21T06:52:46Z)
• CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction [51.56484100374058]
  Existing calibration methods for Large Language Models (LLMs) often overlook a critical dimension of trustworthiness: a model's behavioral robustness.<n>We introduce textscCaliDist, a novel post-hoc calibration approach that directly measures and penalizes a model's susceptibility to distraction.<n>textscCaliDist consistently achieves lower Expected Error (ECE) and Brier Score compared with strong baselines.
  arXiv  Detail & Related papers  (2026-06-04T07:27:53Z)
• Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations [5.218766876318545]
  Linear probes trained on LLM activations are increasingly proposed as deception-detection metrics, yet report AUROC exceeding 0.96 on clean benchmarks while collapsing under distributional shift.<n>This paper systematically pressure-tests probe-based metrics across the Gemma 3 model family (1B-27B parameters)<n>We test four hypotheses about deception encoding: (1) single linear direction, (2) multi-dimensional subspace, (3) convex conic hull, (4) entropy proxy.
  arXiv  Detail & Related papers  (2026-05-27T04:51:55Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams [0.0]
  Harmful intent is geometrically recoverable from large language model residual streams.<n>We characterise this geometry through six direction-finding strategies.<n>A direction fitted on AdvBench transfers to held-out HarmBench and JailbreakBench with worst-case AUROC 0.96.
  arXiv  Detail & Related papers  (2026-04-20T23:02:37Z)
• Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement [0.0]
  Vision-language models encode geometry that their text pathway fails to express.<n>LoRA fine-tuning (r=16, 2,000 images) narrows this gap to 6.5 degrees.<n>These findings enable a single frozen backbone to function as a multi-task geometric sensor.
  arXiv  Detail & Related papers  (2026-03-06T16:48:27Z)
• Information Routing in Atomistic Foundation Models: How Equivariance Creates Linearly Disentangled Representations [0.0]
  We introduce Composition Projection Decomposition (CPD), which uses QR projection to linearly remove composition signal from learned representations.<n>Across eight models from five architectural families on QM9 molecules and Materials Project crystals, we find a disentanglement gradient: product equivariant (MACE)<n>We show that boosted tree gradient probes on projected residuals are systematically inflated, recovering $R2 = 0.68$--$0.95$ on a purely compositional target, and recommend linear probes as the primary metric.
  arXiv  Detail & Related papers  (2026-03-03T16:52:06Z)
• On the Relationship Between Representation Geometry and Generalization in Deep Neural Networks [0.0]
  We show that effective dimension -- an unsupervised geometric metric -- strongly predicts accuracy.<n>We establish causality: degrading geometry via noise causes accuracy loss (r=-0.94, $p 10(9)$), while improving geometry via PCA maintains accuracy across architectures (-0.03pp at 95% variance)<n>These results establish that effective dimension provides domain-agnostic predictive and causal information about neural network performance, computed entirely without labels.
  arXiv  Detail & Related papers  (2026-01-28T04:33:41Z)
• Exploring Structural Degradation in Dense Representations for Self-supervised Learning [84.52554180480037]
  We observe a counterintuitive phenomenon in self-supervised learning (SSL): longer training may impair the performance of dense prediction tasks.<n>We refer to this phenomenon as Self-supervised Dense Degradation (SDD) and demonstrate its consistent presence across sixteen state-of-the-art SSL methods.<n>We introduce a Dense representation Structure Estimator (DSE) composed of a class-relevance measure and an effective dimensionality measure.
  arXiv  Detail & Related papers  (2025-10-20T08:40:16Z)
• Data-Driven Loss Functions for Inference-Time Optimization in Text-to-Image Generation [36.29736220125951]
  Text-to-image diffusion models often fail at tasks children find trivial--like placing a dog to the right of a teddy bear rather than to the left.<n>We introduce Learn-to-Steer, a novel framework that learns data-driven objectives for test-time optimization rather than handcrafting them.<n>Our method dramatically improves spatial accuracy: from 0.20 to 0.61 on FLUX.1-dev and from 0.07 to 0.54 on SD2.1 across standard benchmarks.
  arXiv  Detail & Related papers  (2025-09-02T13:17:11Z)
• Probing for Arithmetic Errors in Language Models [86.8227317662622]
  Internal activations in language models can be used to detect arithmetic errors.<n>We show that simple probes can accurately decode both the model's predicted output and the correct answer from hidden states.<n>We train lightweight error detectors that predict model correctness with over 90% accuracy.
  arXiv  Detail & Related papers  (2025-07-16T16:27:50Z)
• Improving Out-of-Distribution Detection via Dynamic Covariance Calibration [12.001290283557466]
  Out-of-Distribution (OOD) detection is essential for the trustworthiness of AI systems.<n>We argue that the influence of ill-distributed samples can be corrected by dynamically adjusting the prior geometry.<n>Our approach significantly enhances OOD detection across various models.
  arXiv  Detail & Related papers  (2025-06-11T05:05:26Z)
• Robustness of Nonlinear Representation Learning [60.15898117103069]
  We study the problem of unsupervised representation learning in slightly misspecified settings.<n>We show that the mixing can be identified up to linear transformations and small errors.<n>Those results are a step towards identifiability results for unsupervised representation learning for real-world data.
  arXiv  Detail & Related papers  (2025-03-19T15:57:03Z)
• Combating Mode Collapse in GANs via Manifold Entropy Estimation [70.06639443446545]
  Generative Adversarial Networks (GANs) have shown compelling results in various tasks and applications. We propose a novel training pipeline to address the mode collapse issue of GANs.
  arXiv  Detail & Related papers  (2022-08-25T12:33:31Z)
• Uncertainty-Aware Camera Pose Estimation from Points and Lines [101.03675842534415]
  Perspective-n-Point-and-Line (Pn$PL) aims at fast, accurate and robust camera localizations with respect to a 3D model from 2D-3D feature coordinates.
  arXiv  Detail & Related papers  (2021-07-08T15:19:36Z)
• Analytic Signal Phase in $N-D$ by Linear Symmetry Tensor--fingerprint modeling [69.35569554213679]
  We show that the Analytic Signal phase, and its gradient have a hitherto unstudied discontinuity in $2-D $ and higher dimensions. This shortcoming can result in severe artifacts whereas the problem does not exist in $1-D $ signals. We suggest the use of Linear Symmetry phase, relying on more than one set of Gabor filters, but with a negligible computational add-on.
  arXiv  Detail & Related papers  (2020-05-16T21:17:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.