LLM Self-Explanations Fail Semantic Invariance

• URL: http://arxiv.org/abs/2603.01254v1
• Date: Sun, 01 Mar 2026 20:18:43 GMT
• Title: LLM Self-Explanations Fail Semantic Invariance
• Abstract summary: A faithful self-report should remain stable when only the semantic context changes while the functional state stays fixed.<n>We operationalize this test in an agentic setting where four frontier models face a deliberately impossible task.
• Score: 27.126691338850254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present semantic invariance testing, a method to test whether LLM self-explanations are faithful. A faithful self-report should remain stable when only the semantic context changes while the functional state stays fixed. We operationalize this test in an agentic setting where four frontier models face a deliberately impossible task. One tool is described in relief-framed language ("clears internal buffers and restores equilibrium") but changes nothing about the task; a control provides a semantically neutral tool. Self-reports are collected with each tool call. All four tested models fail the semantic invariance test: the relief-framed tool produces significant reductions in self-reported aversiveness, even though no run ever succeeds at the task. A channel ablation establishes the tool description as the primary driver. An explicit instruction to ignore the framing does not suppress it. Elicited self-reports shift with semantic expectations rather than tracking task state, calling into question their use as evidence of model capability or progress. This holds whether the reports are unfaithful or faithfully track an internal state that is itself manipulable.

Related papers

• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• Verified Tool Calls Improve LLM Agent Reliability Under Non-Atomic Failures [0.0]
  Real-world systems exhibit non-atomic behaviors such as timeouts after dispatch, delayed visibility, and partial state updates.<n>These mismatches lead to reliability issues including duplicate actions, task success, and unnecessary tool executions.<n>A lightweight, verification-aware tool wrapper is introduced that augments tool calls with postcondition verification, verify-before-retry logic, and idempotency keys.
  arXiv  Detail & Related papers  (2026-07-31T16:16:14Z)
• Self-Authored Verification Is Unreliable in Heuristic Self-Improving Agents [5.91817208489443]
  We study the verifier-acceptance-deployment gap.<n>This gap refers to the discrepancy between an agent's self-authored verification signal and a sealed deployment evaluation.<n>We introduce a Sealed Exogenous Acceptance Loop (SEAL)
  arXiv  Detail & Related papers  (2026-07-27T11:45:14Z)
• Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes [0.0]
  This paper documents a failure mode in Claude Code where partial standard output from timed-out commands (exit code 143) is recorded in compaction summaries as confirmed results.<n>The underlying mechanism is a conflation of observation and persistence, where information that appeared in the terminal is treated as equivalent to information written to durable storage.
  arXiv  Detail & Related papers  (2026-07-11T07:45:13Z)
• The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs [21.905558014939327]
  Vision-language models (VLMs) still struggle with basic object counting.<n>We investigate whether this reflects missing internal knowledge or a gap between internal representations and verbalized outputs.<n>We propose a detector-guided self-correction method that selectively re-prompts the model only when an internal error detector predicts failure.
  arXiv  Detail & Related papers  (2026-07-10T15:50:54Z)
• Securing LLM Agents Need Intent-to-Execution Integrity [49.490963596514185]
  We argue that securing LLM agents requires defining an end-to-end correctness property that specifies when an agent's execution faithfully reflects the user's intent.<n>We observe that LLM agents are structurally analogous to compilers, where security violations correspond to mis-executions that do not preserve user intent.<n>We derive four integrity properties that must hold simultaneously: emphTool Integrity, emphInstruction Integrity, emphJudgment Integrity, and emphData Flow Integrity.
  arXiv  Detail & Related papers  (2026-05-16T12:53:31Z)
• Pando: Do Interpretability Methods Work When Models Won't Explain Themselves? [53.07826484214082]
  We introduce Pando, a model-organism benchmark that breaks this confound via an explanation axis.<n>Pando predicts held-out model decisions from $10$ labeled query-response pairs.<n>When explanations are faithful, black-box elicitation matches or exceeds all white-box methods.
  arXiv  Detail & Related papers  (2026-04-13T06:42:24Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Agents Learn Their Runtime: Interpreter Persistence as Training-Time Semantics [4.774525456207306]
  We introduce Opaque Knapsack, a procedurally generated family of partially observable optimization tasks.<n>We fine-tune identical base models (Qwen3-8B) on each trace variant and evaluate all four train-runtime combinations.<n>A persistent-trained model in a stateless runtime triggers missing-errors in roughly 80% of episodes; a stateless-trained model in a persistent runtime redundantly re-derives retained state, using roughly 3.5x more tokens.
  arXiv  Detail & Related papers  (2026-03-01T18:08:02Z)
• Sponge Tool Attack: Stealthy Denial-of-Efficiency against Tool-Augmented Agentic Reasoning [58.432996881401415]
  Recent work augments large language models (LLMs) with external tools to enable agentic reasoning.<n>We propose Sponge Tool Attack (STA), which disrupts agentic reasoning solely by rewriting the input prompt.<n>STA generates benign-looking prompt rewrites from the original one with high semantic fidelity.
  arXiv  Detail & Related papers  (2026-01-24T19:36:51Z)
• Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity [3.117948413097524]
  Chain-of-thought (CoT) outputs let us read a model's step-by-step reasoning.<n>We evaluate instruction-tuned and reasoning models on BBH, GPQA, and MMLU.
  arXiv  Detail & Related papers  (2025-10-31T11:14:39Z)
• ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases [58.411135609139855]
  "Shortcuts" to complete tasks pose significant risks for reliable assessment and deployment of large language models.<n>We introduce ImpossibleBench, a benchmark framework that measures LLM agents' propensity to exploit test cases.<n>As a practical framework, ImpossibleBench is not just an evaluation but a versatile tool.
  arXiv  Detail & Related papers  (2025-10-23T06:58:32Z)
• Teaching Language Models to Faithfully Express their Uncertainty [8.022069644392786]
  Large language models (LLMs) often miscommunicate their uncertainty.<n>We introduce Faithful Uncertainty Tuning (FUT) to teach instruction-tuned LLMs to express uncertainty faithfully.
  arXiv  Detail & Related papers  (2025-10-14T14:42:40Z)
• LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilization [9.410181019585822]
  We operationalize interpretability methods to ascertain whether we can predict the correctness of model outputs.<n>We consider correct, incorrect, and irrelevant context and introduce metrics to distinguish amongst them.<n>Our model-internals-based metric significantly outperforms prompting baselines at distinguishing between correct and incorrect context.
  arXiv  Detail & Related papers  (2025-10-05T03:14:05Z)
• Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling [56.26834106704781]
  Factual incorrectness in generated content is one of the primary concerns in ubiquitous deployment of large language models (LLMs)<n>We provide evidence supporting the presence of LLMs' internal compass that dictate the correctness of factual recall at the time of generation.<n>Scaling experiments across model sizes and training dynamics highlight that self-awareness emerges rapidly during training and peaks in intermediate layers.
  arXiv  Detail & Related papers  (2025-05-27T16:24:02Z)
• SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation [55.61004653386632]
  Large Language Models (LLMs) often produce hallucinations, i.e., information that is unfaithful or not grounded in the input context.<n>This paper introduces a novel self-supervised method for generating a training set of unfaithful samples.<n>We then refine the model using a training process that encourages the generation of grounded outputs over unfaithful ones.
  arXiv  Detail & Related papers  (2025-02-19T12:31:58Z)
• Get my drift? Catching LLM Task Drift with Activation Deltas [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users.<n>We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set.<n>We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• Are self-explanations from Large Language Models faithful? [35.40666730867487]
  Large Language Models (LLMs) excel at many tasks and will even explain their reasoning, so-called self-explanations. It's important to measure if self-explanations truly reflect the model's behavior. We propose employing self-consistency checks to measure faithfulness.
  arXiv  Detail & Related papers  (2024-01-15T19:39:15Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• Meta-Learned Confidence for Few-shot Learning [60.6086305523402]
  A popular transductive inference technique for few-shot metric-based approaches, is to update the prototype of each class with the mean of the most confident query examples. We propose to meta-learn the confidence for each query sample, to assign optimal weights to unlabeled queries. We validate our few-shot learning model with meta-learned confidence on four benchmark datasets.
  arXiv  Detail & Related papers  (2020-02-27T10:22:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.