Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations

• URL: http://arxiv.org/abs/2603.18353v1
• Date: Wed, 18 Mar 2026 23:31:05 GMT
• Title: Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations
• Abstract summary: Language models encode task-relevant knowledge in internal representations that far exceeds their output performance.<n>Current mechanistic interpretability methods cannot reliably translate internal knowledge into corrected outputs.
• Score: 1.6676380665893165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models encode task-relevant knowledge in internal representations that far exceeds their output performance, but whether mechanistic interpretability methods can bridge this knowledge-action gap has not been systematically tested. We compared four mechanistic interpretability methods -- concept bottleneck steering (Steerling-8B), sparse autoencoder feature steering, logit lens with activation patching, and linear probing with truthfulness separator vector steering (Qwen 2.5 7B Instruct) -- for correcting false-negative triage errors using 400 physician-adjudicated clinical vignettes (144 hazards, 256 benign). Linear probes discriminated hazardous from benign cases with 98.2% AUROC, yet the model's output sensitivity was only 45.1%, a 53-percentage-point knowledge-action gap. Concept bottleneck steering corrected 20% of missed hazards but disrupted 53% of correct detections, indistinguishable from random perturbation (p=0.84). SAE feature steering produced zero effect despite 3,695 significant features. TSV steering at high strength corrected 24% of missed hazards while disrupting 6% of correct detections, but left 76% of errors uncorrected. Current mechanistic interpretability methods cannot reliably translate internal knowledge into corrected outputs, with implications for AI safety frameworks that assume interpretability enables effective error correction.

Related papers

• Traffic-Aware Randomized Smoothing for LLM-Based Network Intrusion Detection [1.4915002678801434]
  We present Traffic-Aware Randomized Smoothing (TA-RS) for intrusion detection systems.<n>It is a certified defense that injects Gaussian noise exclusively into the controllable (DC) subspace.<n>It achieves 55-100% certified accuracy on two datasets.
  arXiv  Detail & Related papers  (2026-07-15T13:08:19Z)
• LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data [2.1443570696048906]
  Large language models (LLMs) are increasingly applied to structured clinical data.<n>We compare Qwen 2.5 7B and XGBoost on a prediction task via attribution divergence analysis.
  arXiv  Detail & Related papers  (2026-06-17T18:49:44Z)
• Discovery of Hidden Miscalibration Regimes [52.452902154360565]
  Models may be systematically overconfident on some kinds of inputs and underconfident on others.<n>We define the corresponding miscalibration field and propose a diagnostic framework for estimating it.<n>Our approach learns a calibration-aware representation of the input space and estimates signed local miscalibration by kernel smoothing in the learned geometry.
  arXiv  Detail & Related papers  (2026-05-13T13:07:50Z)
• Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimes [4.738949927143789]
  We investigate whether linearly decodable failure signals in hidden states can be leveraged to correct those failures.<n>We find that decodable failure structure supports post-generation reliability estimation even when the fixed linear steering family cannot exploit it for correction.
  arXiv  Detail & Related papers  (2026-05-07T05:58:38Z)
• Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders [10.172598963520961]
  Large language models can be uncertain yet correct, or confident yet wrong, raising the question of whether their output-level uncertainty and their actual correctness are driven by the same internal mechanisms or by distinct feature populations.<n>We introduce a 2x2 framework that partitions model predictions along correctness and confidence axes, and uses sparse autoencoders to identify features associated with each dimension independently.
  arXiv  Detail & Related papers  (2026-04-21T20:34:14Z)
• Severe Domain Shift in Skeleton-Based Action Recognition:A Study of Uncertainty Failure in Real-World Gym Environments [0.0]
  We present a systematic study of a severe domain shift using a novel Gym2D dataset and the UCF101 dataset.<n>Our work challenges standard deployment assumptions, providing a principled safety analysis of both semantic and geometric skeleton recognition deployment.
  arXiv  Detail & Related papers  (2026-03-16T17:37:17Z)
• TRIAGE: Type-Routed Interventions via Aleatoric-Epistemic Gated Estimation in Robotic Manipulation and Adaptive Perception -- Don't Treat All Uncertainty the Same [2.755751829139168]
  Most uncertainty-aware robotic systems collapse prediction uncertainty into a single scalar score and use it to trigger uniform corrective responses.<n>This aggregation obscures whether uncertainty arises from corrupted observations or from mismatch between the learned model and the true system dynamics.<n>We introduce a lightweight post hoc framework that decomposes uncertainty into aleatoric and epistemic components and uses these signals to regulate system responses at inference time.
  arXiv  Detail & Related papers  (2026-03-09T09:07:43Z)
• Probabilistic Soundness Guarantees in LLM Reasoning Chains [37.440902632372904]
  Autoregressive Reasoning Entailment Stability (ARES) is a probabilistic framework that evaluates each reasoning step based solely on previously-verified premises.<n>ARES achieves state-of-the-art performance across four benchmarks and demonstrates superior robustness on very long synthetic reasoning chains.
  arXiv  Detail & Related papers  (2025-07-17T09:40:56Z)
• Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models [0.7910367295422812]
  Large language models (LLMs) make mistakes and can explore unproductive reasoning paths.<n>Self-correction capability is essential for deploying LLMs in safety-critical applications.<n>We uncover a systematic failure: LLMs cannot correct errors in their own outputs while successfully correcting identical errors from external sources.
  arXiv  Detail & Related papers  (2025-07-03T16:41:30Z)
• A Coin Has Two Sides: A Novel Detector-Corrector Framework for Chinese Spelling Correction [79.52464132360618]
  Chinese Spelling Correction (CSC) stands as a foundational Natural Language Processing (NLP) task. We introduce a novel approach based on error detector-corrector framework. Our detector is designed to yield two error detection results, each characterized by high precision and recall.
  arXiv  Detail & Related papers  (2024-09-06T09:26:45Z)
• Improving Uncertainty-Error Correspondence in Deep Bayesian Medical Image Segmentation [3.3572047447192626]
  We train the FlipOut model with the Accuracy-vs-Uncertainty (AvU) loss which promotes uncertainty to be present only in inaccurate regions. We apply this method on datasets of two radiotherapy body sites, c.f. head-and-neck CT and prostate MR scans.
  arXiv  Detail & Related papers  (2024-09-05T12:31:51Z)
• Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis [127.85293480405082]
  The rapid development of large language models (LLMs) has not only provided numerous opportunities but also presented significant challenges. Existing alignment methods usually direct LLMs toward the favorable outcomes by utilizing human-annotated, flawless instruction-response pairs. This study proposes a novel alignment technique based on mistake analysis, which deliberately exposes LLMs to erroneous content to learn the reasons for mistakes and how to avoid them.
  arXiv  Detail & Related papers  (2023-10-16T14:59:10Z)
• Conservative Prediction via Data-Driven Confidence Minimization [70.93946578046003]
  In safety-critical applications of machine learning, it is often desirable for a model to be conservative. We propose the Data-Driven Confidence Minimization framework, which minimizes confidence on an uncertainty dataset.
  arXiv  Detail & Related papers  (2023-06-08T07:05:36Z)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition [109.62614226793833]
  The trade-off between robustness and accuracy has been widely studied in the adversarial literature. We find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty.
  arXiv  Detail & Related papers  (2022-02-21T10:36:09Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)
• Localization Uncertainty Estimation for Anchor-Free Object Detection [48.931731695431374]
  There are several limitations of the existing uncertainty estimation methods for anchor-based object detection. We propose a new localization uncertainty estimation method called UAD for anchor-free object detection. Our method captures the uncertainty in four directions of box offsets that are homogeneous, so that it can tell which direction is uncertain.
  arXiv  Detail & Related papers  (2020-06-28T13:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.