Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations

• URL: http://arxiv.org/abs/2607.20379v1
• Date: Wed, 22 Jul 2026 17:10:23 GMT
• Title: Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations
• Abstract summary: We show the test is passed in two ways, neither faithful.<n>On a released Qwen-2.5-7B verbalizer, explanations reconstruct well above chance.<n>For AI safety, RECAP makes designated internal content independently checkable.
• Score: 0.11280931253550518
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural-language autoencoders score explanations of hidden activations by reconstruction: an explanation is deemed faithful if the activation can be regenerated from it. The test is structurally insensitive to individual false claims: if flipping a claim does not change the reconstruction, the claim is never penalized. We show the test is passed in two ways, neither faithful. On a released Qwen-2.5-7B verbalizer, explanations reconstruct well above chance while ~2% of specific claims are reconstruction-dependent, so the score tracks gist, not specific facts. Under exact synthetic ground truth, the standard recipe develops co-adapted private codes (false wording the reconstruction depends on) in 5/5 runs, and fixes that leave the target model unchanged do not help. We contribute two audit protocols, the grounded-vs-true cross and the evaluator swap, and RECAP (Readable Encodings via Co-trained Auxiliary Predictors): linear heads trained alongside the target model to keep designated content decodable. On RECAP-trained sandbox models, fresh verbalizers state the designated content truly and the codes vanish, at a +0.001-nat cost. This replicates on a pretrained Pythia-160M: the content becomes reliably probe-decodable, though a fresh verbalizer conveys it only in part (truth 0.44-0.46 vs a near-zero control). For interpretability, high reconstruction does not certify individual claims. For AI safety, RECAP makes designated internal content independently checkable against probes rather than asserted by prose a model can game: an independent probe scores the verbalizer's true claims above its false ones (AUC 0.96, vs 0.82 without RECAP). Against an adversary that edits an explanation to maximize the reconstruction score while lying (suppressing ~87% of its lie penalty), the RECAP probe still flags the lies (AUC 0.95) while the control probe collapses to chance (0.51).

Related papers

• Reversibility-Verified De-identification for Cloud-Local LLM Inference: A Locally Certified Dehydrate-Rehydrate Loop with Layered Assurance (DR-SL) [1.802678437602184]
  DR-SL (Dehydrate-Rehydrate with Self-Learning loop) formalizes de-identification completeness as two measurable conditions.<n>A fully local two-branch verifier iterates dehydration under a lexicographic gate with guaranteed termination.<n>We prove Fano-type lower bounds, a Pufferfish witness, and a rate-privacy feasibility criterion, and state their scope plainly.
  arXiv  Detail & Related papers  (2026-09-14T01:04:58Z)
• VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement [57.86962208273888]
  We present an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations.<n>Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state.<n>We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time.
  arXiv  Detail & Related papers  (2026-09-02T20:36:45Z)
• Calibrating Criterion Revision in LLM Agents: Failure Modes and a Trace-Anchored Protocol [3.202978695204522]
  Language-model agents can improve after failure or carry text across episodes without revising what counts as success.<n>We evaluate CMB-0.1 on twelve cross-domain cases and four arms.<n>No model trial satisfies all five conditions, but this zero does not establish general capability absence.
  arXiv  Detail & Related papers  (2026-08-21T04:21:28Z)
• Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Models [4.1626636325601405]
  Our defense, decoy hardening, concedes the refusal strip and poisons its payoff.<n>We instantiate it on seven models from five families (9B-122B, dense and mixture-of-experts)
  arXiv  Detail & Related papers  (2026-08-17T23:26:32Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs [4.721990925113432]
  Aligned language models routinely misreport under non-evidential pressure.<n>We study the two demands, resist (ignore forbidden pressure) and update (follow licensed evidence)<n>We causally localize low-rank report coordinates for answer, confidence, and caveat.
  arXiv  Detail & Related papers  (2026-07-14T17:28:25Z)
• Agent-Safety Evaluations as Load-Bearing Evidence: A Vendor-Neutral, Cross-Harness Reconstructability Metric [0.0]
  No vendor-neutral, runnable instrument scores reconstructability as an evaluation-validity metric.<n>This paper introduces a property-level reconstructability metric over eight decision-property classes.
  arXiv  Detail & Related papers  (2026-07-14T07:54:37Z)
• LatentRevise: Learning from Zero-Hit Reasoning [46.1536368072511]
  Reinforcement learning with verifiable rewards is bottlenecked by hard prompts on which correct trajectories have low probability.<n>We frame such zero-hit prompts as RLVR's sampling frontier, where new reasoning behavior is most valuable yet least likely to be sampled.<n>We introduce LatentRevise, a first-order latent revision method that recovers training signal for this zero-hit regime.
  arXiv  Detail & Related papers  (2026-06-29T08:14:47Z)
• Let the Results Speak: A Replication-First Paradigm for LLM Behavioral Benchmarking [22.825786049667602]
  We propose a replication-first paradigm to anchor validity to a single human-rater consensus.<n>We certify the instrument via four properties -- reliability across K runs, cross-instrument replication across architecturally distinct judges, historical-footprint calibration via judges from earlier training cohorts, and pre-registered prediction.<n>We test it on emotional accompaniment by letting the self-evolve data-driven across: the dimensions are not pre-stipulated and the procedure stabilizes to a 9-dimension set.
  arXiv  Detail & Related papers  (2026-05-27T03:41:11Z)
• CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolution [52.691495954442985]
  CoVerRL is a framework where a single model alternates between generator and verifier roles, with each capability bootstrapping the other.<n> Experiments across Qwen and Llama model families demonstrate that CoVerRL outperforms label-free baselines by 4.7-5.9% on mathematical reasoning benchmarks.<n>Self-verification accuracy improves from around 55% to over 85%, confirming that both capabilities genuinely co-evolve.
  arXiv  Detail & Related papers  (2026-03-18T14:38:55Z)
• Probing for Knowledge Attribution in Large Language Models [45.47366023067617]
  Large language models (LLMs) often generate fluent but unfounded claims, or hallucinations.<n> Proper mitigation depends on knowing whether a model's answer is based on the prompt or its internal weights.<n>We show that a probe, a simple linear classifier trained on model hidden representations, can reliably predict contributive attribution.
  arXiv  Detail & Related papers  (2026-02-26T09:21:12Z)
• IR$^3$: Contrastive Inverse Reinforcement Learning for Interpretable Detection and Mitigation of Reward Hacking [67.20568716300272]
  Reinforcement Learning from Human Feedback (RLHF) enables powerful LLM alignment but can introduce reward hacking.<n>We introduce IR3 (Interpretable Reward Reconstruction and Rectification), a framework that reverse-engineers, interprets, and surgically repairs the implicit objectives driving RLHF-tuned models.<n>We show that IR3 achieves 0.89 correlation with ground-truth rewards, identifies hacking features with over 90% precision, and significantly reduces hacking behaviors while maintaining capabilities within 3% of the original model.
  arXiv  Detail & Related papers  (2026-02-23T01:14:53Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation [12.503662455234954]
  We show that modern language models produce confident hallucinations even when wrong answers carry catastrophic consequences.<n>We propose Reinforced Hesitation (RH): a modification to Reinforcement Learning from Verifiable Rewards (RLVR) to use ternary rewards instead of binary.
  arXiv  Detail & Related papers  (2025-11-14T17:20:45Z)
• A Granular Study of Safety Pretraining under Model Abliteration [64.24346997570275]
  We study model abliteration, a lightweight projection technique designed to remove refusal-sensitive directions.<n>We issue 100 prompts with balanced harmful and harmless cases, classify responses as **Refusal** or **Non-Refusal** using multiple judges, and validate judge fidelity.<n>Our study produces a checkpoint-level characterization of which data-centric safety components remain robust under abliteration.
  arXiv  Detail & Related papers  (2025-10-03T07:01:45Z)
• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.