How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals

• URL: http://arxiv.org/abs/2604.22271v2
• Date: Fri, 01 May 2026 09:11:21 GMT
• Title: How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals
• Abstract summary: Large language models can detect their own errors and sometimes correct them without external feedback.<n>We investigate this through the lens of second-order models of confidence from decision neuroscience.
• Score: 6.467495925520036
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models can detect their own errors and sometimes correct them without external feedback, but the underlying mechanisms remain unknown. We investigate this through the lens of second-order models of confidence from decision neuroscience. In a first-order system, confidence derives from the generation signal itself and is therefore maximal for the chosen response, precluding error detection. Second-order models posit a partially independent evaluative signal that can disagree with the committed response, providing the basis for error detection. Kumaran et al. (2026) showed that LLMs cache a confidence representation at a token immediately following the answer (i.e. post-answer newline: PANL) -- that causally drives verbal confidence and dissociates from log-probabilities. Here we test whether this PANL signal extends beyond confidence to support error detection and self-correction. Here we test whether this signal supports error detection and self-correction, deriving predictions from the second-order framework. Using a verify-then-correct paradigm, we show that: (i) verbal confidence predicts error detection far beyond token log-probabilities, ruling out a first-order account; (ii) PANL activations predict error detection beyond verbal confidence itself; and (iii) PANL predicts which errors the model can correct -- where all behavioural signals fail. Causal interventions confirm that PANL signals rescue error detection behavior when answer information is corrupted. All findings replicate across models (Gemma 3 27B and Qwen 2.5 7B) and tasks (TriviaQA and MNLI). These results reveal that LLMs naturally implement a second-order confidence architecture whose internal evaluative signal encodes not only whether an answer is likely wrong but whether the model has the knowledge to fix it.

Related papers

• On the Robustness of LLMs' Internal Representation of Code Correctness [0.3653697742557465]
  Research shows that models' own confidence signals are poorly calibrated with actual correctness.<n>Recent work captured an internal signal of code correctness that is able to judge candidate solutions better than the model's token-level or stated confidence.<n>We study this question systematically, varying how the signal is extracted from the model internals.
  arXiv  Detail & Related papers  (2026-08-08T17:56:31Z)
• The Knowing-Saying Gap: When Probes See Errors that Confidence Misses [20.46211358172975]
  Linear probes detect corrupted context in language models with near-perfect accuracy, yet this does not translate into reliable failure prediction.<n>Across multi-hop arithmetic chains, probes that detect corruption turn out to be uninformative about final answer correctness.<n>As a real-time monitor, probe-based interventions are sharply model and error-type dependent.
  arXiv  Detail & Related papers  (2026-07-21T12:13:10Z)
• LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling [26.999207995495354]
  We ask whether large language models (LLMs) possess latent metacognitive ability that can be turned into effective test-time control.<n>Inspired by the Nelson--Narens theory from cognitive psychology, we propose a metacognitive harness that separates monitoring from reasoning.
  arXiv  Detail & Related papers  (2026-05-13T23:09:25Z)
• Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal [6.908637308550535]
  Chain-of-thought assumes that generated reasoning reflects a model's internal computation.<n>We show this assumption is wrong in a specific, measurable way.<n>Models internally detect their own reasoning errors but outwardly express confidence in them.
  arXiv  Detail & Related papers  (2026-05-10T12:26:46Z)
• Closing the Confidence-Faithfulness Gap in Large Language Models [3.122242683664973]
  Large language models verbalize confidence scores that are largely detached from their actual accuracy.<n>We present a mechanistic interpretability analysis of verbalized confidence.<n>We introduce a two-stage adaptive steering pipeline that reads the model's internal accuracy estimate and steers verbalized output to match it.
  arXiv  Detail & Related papers  (2026-03-26T05:42:04Z)
• Decoding the Critique Mechanism in Large Reasoning Models [50.821607345799386]
  Large Reasoning Models (LRMs) exhibit backtracking and self-verification mechanisms that enable them to revise intermediate steps and reach correct solutions.<n>We investigate how current LRMs recover from errors by inserting arithmetic mistakes in their intermediate reasoning steps.<n>We find that despite the error propagating through the chain-of-thought, the model still reaches the correct final answer.
  arXiv  Detail & Related papers  (2026-03-17T10:03:30Z)
• Can LLMs Detect Their Confabulations? Estimating Reliability in Uncertainty-Aware Language Models [24.72990207218907]
  Large Language Models (LLMs) are prone to generating fluent but incorrect content, known as confabulation.<n>We investigate how in-context information influences model behavior and whether LLMs can identify their unreliable responses.
  arXiv  Detail & Related papers  (2025-08-11T16:12:36Z)
• Probabilistic Soundness Guarantees in LLM Reasoning Chains [37.440902632372904]
  Autoregressive Reasoning Entailment Stability (ARES) is a probabilistic framework that evaluates each reasoning step based solely on previously-verified premises.<n>ARES achieves state-of-the-art performance across four benchmarks and demonstrates superior robustness on very long synthetic reasoning chains.
  arXiv  Detail & Related papers  (2025-07-17T09:40:56Z)
• Probing for Arithmetic Errors in Language Models [86.8227317662622]
  Internal activations in language models can be used to detect arithmetic errors.<n>We show that simple probes can accurately decode both the model's predicted output and the correct answer from hidden states.<n>We train lightweight error detectors that predict model correctness with over 90% accuracy.
  arXiv  Detail & Related papers  (2025-07-16T16:27:50Z)
• Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs [87.79350168490475]
  This work formally defines self-consistent errors and evaluates mainstream detection methods on them.<n>All four types of detection methods significantly struggle to detect self-consistent errors.<n>Motivated by the observation that self-consistent errors often differ across LLMs, we propose a simple but effective cross-model probe.
  arXiv  Detail & Related papers  (2025-05-23T09:18:56Z)
• Improving Label Error Detection and Elimination with Uncertainty Quantification [5.184615738004059]
  We develop novel, model-agnostic algorithms for Uncertainty Quantification-Based Label Error Detection (UQ-LED) Our UQ-LED algorithms outperform state-of-the-art confident learning in identifying label errors. We propose a novel approach to generate realistic, class-dependent label errors synthetically.
  arXiv  Detail & Related papers  (2024-05-15T15:17:52Z)
• Improving Uncertainty Calibration via Prior Augmented Data [56.88185136509654]
  Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. They are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. We propose a solution by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels.
  arXiv  Detail & Related papers  (2021-02-22T07:02:37Z)
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering [80.82194311274694]
  We examine the question "how can we know when language models know, with confidence, the answer to a particular query?" We examine three strong generative models -- T5, BART, and GPT-2 -- and study whether their probabilities on QA tasks are well calibrated. We then examine methods to calibrate such models to make their confidence scores correlate better with the likelihood of correctness.
  arXiv  Detail & Related papers  (2020-12-02T03:53:13Z)
• Probing Model Signal-Awareness via Prediction-Preserving Input Minimization [67.62847721118142]
  We evaluate models' ability to capture the correct vulnerability signals to produce their predictions. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR) The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric.
  arXiv  Detail & Related papers  (2020-11-25T20:05:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.