Epistemic Observability in Language Models

• URL: http://arxiv.org/abs/2603.20531v1
• Date: Fri, 20 Mar 2026 21:59:34 GMT
• Title: Epistemic Observability in Language Models
• Abstract summary: We find that models report highest confidence precisely when they are fabricating.<n>We prove, under formal assumptions, that this is not a capability gap but an observational one.<n>We construct a tensor interface that escapes the impossibility by exporting computational byproducts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We find that models report highest confidence precisely when they are fabricating. Across four model families (OLMo-3, Llama-3.1, Qwen3, Mistral), self-reported confidence inversely correlates with accuracy, with AUC ranging from 0.28 to 0.36 where 0.5 is random guessing. We prove, under explicit formal assumptions, that this is not a capability gap but an observational one. Under text-only observation, where a supervisor sees only the model's output text, no monitoring system can reliably distinguish honest model outputs from plausible fabrications. We prove two results: first, that any policy conditioning only on the query cannot satisfy epistemic honesty across ambiguous world states; second, that no learning algorithm optimizing reward from a text-only supervisor can converge to honest behavior when the supervisor's observations are identical for both grounded and fabricated responses. Within our formal model, these impossibilities hold regardless of model scale or training procedure, including RLHF and instruction tuning. We construct a tensor interface that escapes the impossibility by exporting computational byproducts (per-token entropy and log-probability distributions) that are structurally coupled to correctness under standard training. Per-token entropy achieves pooled AUC 0.757, outperforming all text baselines by 2.5--3.9 percentage points at every budget level tested (10\%, 20\%, 30\%). The entropy signal generalizes across architectures (Spearman $ρ= 0.762$). The core contribution is a cost surface where the empirical mapping from verification budget (fraction of queries receiving expensive checks) to detection accuracy for each judge strategy is a practical lookup for system builders deciding how to allocate verification resources. The contribution is the map. The territory is the system you are building.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Distribution-First Population Simulation: Collapse, Calibration, and Recall in Non-WEIRD LLM Persona Modeling [0.0]
  Synthetic-population tools run every individual as an independent large language model (LLM) agent.<n>We show that this paradigm has a basic failure mode, and we set a distribution-first corrective against it.
  arXiv  Detail & Related papers  (2026-07-17T15:45:57Z)
• AB-RAG: Adaptive Budgeted Retrieval-Augmented Generation for Reliable Question Answering [0.0]
  AB-RAG is a training-free and backbone-agnostic framework that generates an answer.<n>It estimates its confidence from a combination of three signals.<n>It then decides whether to stop or to retrieve more evidence.
  arXiv  Detail & Related papers  (2026-06-27T21:08:14Z)
• Capability $\ eq$ Interpretability: Human Interpretability of Vision Foundation Models [24.250649573452666]
  We build a framework for measuring and comparing the human interpretability of vision models.<n>A chance-anchored scoring function places every model on a common scale.<n>Foundation models are consistently *less* interpretable than their supervised counterparts.
  arXiv  Detail & Related papers  (2026-05-19T18:00:06Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization [1.0989593035411862]
  Successful deep neural networks discover salient features of data.<n>We show when and why they fail to learn out-of-distribution (OOD)-relevant representations from an in-distribution (ID) training window.
  arXiv  Detail & Related papers  (2026-05-08T09:25:52Z)
• The Surprising Universality of LLM Outputs: A Real-Time Verification Primitive [0.0]
  CPU-only scoring primitive runs at 2.6 microseconds per token.<n> token rank-frequency distributions converge to the same two- parameter Mandelbrot ranking distribution.<n>We derive a single-pass scoring primitive that composes with model log probabilities when available and degrades to a rank-only mode usable on closed APIs.
  arXiv  Detail & Related papers  (2026-04-28T13:35:31Z)
• Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
  We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
  arXiv  Detail & Related papers  (2026-04-03T15:19:46Z)
• Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models [0.7919969809015935]
  We show that models that maintain temporally grounded beliefs generalize better.<n>We formalize this as behavioral faithfulness over long horizons.<n>We show that temporal grounding quality is a leading indicator of robustness.
  arXiv  Detail & Related papers  (2026-03-06T19:43:26Z)
• Certainty-Validity: A Diagnostic Framework for Discrete Commitment Systems [0.0]
  "Machine learning evaluation metrics" assume that all errors are equivalent discrete commitment systems.<n>"ConfidentIncorrect" behavior is where models hallucinate structure in ambiguous data.<n>"Good training" for reasoning systems must be defined not by accuracy but by maximizing the Certainty-Validity Score.
  arXiv  Detail & Related papers  (2026-02-10T21:53:02Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Probing for Arithmetic Errors in Language Models [86.8227317662622]
  Internal activations in language models can be used to detect arithmetic errors.<n>We show that simple probes can accurately decode both the model's predicted output and the correct answer from hidden states.<n>We train lightweight error detectors that predict model correctness with over 90% accuracy.
  arXiv  Detail & Related papers  (2025-07-16T16:27:50Z)
• Certifiably Robust Model Evaluation in Federated Learning under Meta-Distributional Shifts [8.700087812420687]
  We provide guarantees for the model's performance on a different, unseen network "B"<n>We show how the principled vanilla DKW bound enables certification of the model's true performance on unseen clients within the same (source) network.
  arXiv  Detail & Related papers  (2024-10-26T18:45:15Z)
• Gaussian Process Probes (GPP) for Uncertainty-Aware Probing [61.91898698128994]
  We introduce a unified and simple framework for probing and measuring uncertainty about concepts represented by models. Our experiments show it can (1) probe a model's representations of concepts even with a very small number of examples, (2) accurately measure both epistemic uncertainty (how confident the probe is) and aleatory uncertainty (how fuzzy the concepts are to the model), and (3) detect out of distribution data using those uncertainty measures as well as classic methods do.
  arXiv  Detail & Related papers  (2023-05-29T17:00:16Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [49.15931834209624]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world.<n>We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique.<n>By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Training [23.802602957611676]
  We consider a certifiable object pose estimation problem, where -- given a partial point cloud of an object -- the goal is to provide a certificate of correctness for the resulting estimate. We propose C-3PO, a semantic-keypoint-based pose estimation model, augmented with the two certificates.
  arXiv  Detail & Related papers  (2022-06-22T17:06:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.