Architectural Observability Collapse in Transformers

• URL: http://arxiv.org/abs/2604.24801v2
• Date: Tue, 05 May 2026 07:56:54 GMT
• Title: Architectural Observability Collapse in Transformers
• Abstract summary: Training preserved an internal decision-quality signal that output confidence does not expose.<n>Confidence controls absorb on average 60.3% of raw probe signal across 14 models in 6 families.<n>Training preserved observability where Llama 3.1 8B collapses at identical 32-layer, 32-head, 4096-hidden shape.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Activation monitoring can catch confident errors in autoregressive transformers only if training preserved an internal decision-quality signal that output confidence does not expose. Monitorability is an architectural property before it is a monitor-design problem. We define observability: the linear readability of per-token decision quality from frozen mid-layer activations after controlling for max-softmax confidence and activation norm. Confidence controls absorb on average 60.3% of raw probe signal across 14 models in 6 families. Observability is not a generic property of transformers. In Pythia's controlled suite, all three tested runs at the 24-layer, 16-head configuration collapse to rho_partial ~0.10 across a 3.5x parameter gap and two Pile variants, while six other configurations occupy a separated healthy band from 0.21 to 0.38. The output-controlled residual r_OC collapses at the same points; neither nonlinear probes nor layer sweeps recover healthy-range signal. Checkpoint dynamics localize the cause: both matched-width configurations form the signal at the earliest measured checkpoint, and training erases it in the 1.4B even as it reaches lower final loss than the 1B. Across independent recipes the collapse map changes but the phenomenon persists. Qwen 2.5 and Llama differ by 2.9x at matched 3B scale, with probe-seed distributions that do not overlap. Mistral 7B v0.3 preserves observability where Llama 3.1 8B collapses at identical 32-layer, 32-head, 4096-hidden shape. Within Qwen 2.5, observability persists from 0.5B through 32B. A WikiText-trained observer transfers to downstream QA without task-specific training: at 20% flag rate, exclusive catch reaches 10.9-13.4% in seven of nine model-task cells, near the 12-15% language-modeling ceiling. Architecture selection is a monitoring decision.

Related papers

• K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models [32.7114174898989]
  We study K/V-cache interventions as a structured surface for persona control in decoder-only language models.<n>We report two consistent universalitys between representation-level alignment and behavioral expression, plus a common failure under position perturbations.
  arXiv  Detail & Related papers  (2026-09-10T02:56:50Z)
• Lagged Coupling: Internal Representations Become Readable Before They Become Causal [0.0]
  Internal readability systematically outruns causal efficacy, and the lag does not shrink with scale.<n>A pre-registered OLMo-2 replication preserves the direction at attenuated magnitude.
  arXiv  Detail & Related papers  (2026-09-01T10:46:42Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• A Calibration Audit of Confidence in Feed-Forward 3D Reconstruction [45.496856835431885]
  3D reconstruction models emit per-pixel confidence that downstream systems read as a reliability signal.<n>We score the confidence on four properties, how well it ranks error, whether its level is right on average, whether it holds across the confidence range, and whether its intervals cover the truth.<n>The confidence ranks error well, but the predicted uncertainty is too low when it is read under conditions that are not exactly those of training.
  arXiv  Detail & Related papers  (2026-08-30T10:21:34Z)
• Beyond Control Points: Arcsecond Relative-Motion Estimation of Vision Measurement Platforms With Incomplete or Absent Control Fields [43.43572961146893]
  We develop a control-adaptive differential framework that estimates platform motion directly from image displacements and known 3D points.<n>We derive the rotation observability condition, a leakage bound for unmodeled translation and nonrigid point motion, and the single-point axial-prior bias law.
  arXiv  Detail & Related papers  (2026-08-14T03:41:30Z)
• Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling [0.0]
  We show that when a language model fails on surface-perturbed input, "which layer is responsible" has three natural operationalizations.<n>We identify two propagation regimes - spike-and-suppress and late-accumulation.<n>A fixed-harness layer sweep across four models (3.8-8B) confirms the core prediction on chain-of-thought GSM8K.
  arXiv  Detail & Related papers  (2026-08-04T15:48:55Z)
• Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts [0.0]
  We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.<n>We classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs across three quantization precisions.<n>We find that while accuracy is robust across precisions, Hollow Convergence shows a significant size-dependent shift under NF4.
  arXiv  Detail & Related papers  (2026-07-10T21:55:05Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses [57.20181537213498]
  Agentic Harness Engineering instrumented the three stages of any engineering loop with matched observability pillars.<n>Ten AHE lift pass@1 on Terminal-Bench 2 from 69.7% to 77.0%, surpassing the human-designed harness Codex-CLI.<n>These results position observability-driven evolution as a practical pathway to keep coding-agent harnesses continually improving.
  arXiv  Detail & Related papers  (2026-04-28T16:55:02Z)
• Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams [0.0]
  Harmful intent is geometrically recoverable from large language model residual streams.<n>We characterise this geometry through six direction-finding strategies.<n>A direction fitted on AdvBench transfers to held-out HarmBench and JailbreakBench with worst-case AUROC 0.96.
  arXiv  Detail & Related papers  (2026-04-20T23:02:37Z)
• The Geometric Canary: Predicting Steerability and Detecting Drift via Representational Stability [0.0]
  We show that geometric stability, the consistency of a representation's pairwise distance structure, addresses two functions.<n> unsupervised stability fails entirely for steering on real-world tasks.<n>It excels at drift detection, measuring nearly $2times$ greater geometric change than CKA during post-training alignment.
  arXiv  Detail & Related papers  (2026-04-20T01:24:45Z)
• Disposition Distillation at Small Scale: A Three-Arc Negative Result [0.0]
  Internal draft reported +33.9-point MCAS and +15.3-point HumanEval gains on a Qwen3-0.6B student; a second-pass sanity check falsified both numbers before publication.<n>We find no operator that moves judge-measured disposition without damaging content or collapsing into stylistic mimicry.<n>We contribute a three-arc negative result with mechanism, a two-failure-mode taxonomy for linear h_last probes, and an honest falsification pipeline that converts the class of false positives we ourselves produced into publishable negatives.
  arXiv  Detail & Related papers  (2026-04-13T17:40:31Z)
• LatentAudit: Real-Time White-Box Faithfulness Monitoring for Retrieval-Augmented Generation with Verifiable Deployment [19.317475241300397]
  We introduce LatentAudit, a white-box auditor that pools mid-to-late residual-stream activations from an open-weight generator.<n>We show that residual-stream geometry carries a usable faithfulness signal, that this signal survives architecture changes, and that the same rule remains amenable to public verification.
  arXiv  Detail & Related papers  (2026-04-07T02:55:32Z)
• Reliable Control-Point Selection for Steering Reasoning in Large Language Models [28.288321095634128]
  Steering vectors offer a training-free mechanism for controlling reasoning behaviors in large language models.<n>However, constructing effective vectors requires identifying genuine behavioral signals in the model's hidden states.<n>We show that current methods detect these behaviors through keyword matching in chain-of-thought traces, implicitly assuming that every detected boundary encodes a genuine behavioral signal.<n>We develop a probabilistic model that formalizes intrinsic reasoning behaviors as events with context-dependent trigger probabilities, and show that unstable boundaries dilute the steering signal.
  arXiv  Detail & Related papers  (2026-04-02T14:48:56Z)
• TRIAGE: Type-Routed Interventions via Aleatoric-Epistemic Gated Estimation in Robotic Manipulation and Adaptive Perception -- Don't Treat All Uncertainty the Same [2.755751829139168]
  Most uncertainty-aware robotic systems collapse prediction uncertainty into a single scalar score and use it to trigger uniform corrective responses.<n>This aggregation obscures whether uncertainty arises from corrupted observations or from mismatch between the learned model and the true system dynamics.<n>We introduce a lightweight post hoc framework that decomposes uncertainty into aleatoric and epistemic components and uses these signals to regulate system responses at inference time.
  arXiv  Detail & Related papers  (2026-03-09T09:07:43Z)
• Engineering Verifiable Modularity in Transformers via Per-Layer Supervision [0.0]
  Ablating an attention head identified as critical for capitalization produces minimal behavioral change because distributed redundancy compensates for damage.<n>We demonstrate that architectural interventions can expose hidden modularity.<n>This es tablishes a methodology for transforming interpretability from passive observation to active control.
  arXiv  Detail & Related papers  (2026-03-08T05:18:14Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)
• Beyond calibration: estimating the grouping loss of modern neural networks [68.8204255655161]
  Proper scoring rule theory shows that given the calibration loss, the missing piece to characterize individual errors is the grouping loss. We show that modern neural network architectures in vision and NLP exhibit grouping loss, notably in distribution shifts settings.
  arXiv  Detail & Related papers  (2022-10-28T07:04:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.