Align, Unify, Suppress, Route: A Coherentist View of Transformer Computation

• URL: http://arxiv.org/abs/2608.22034v1
• Date: Sat, 22 Aug 2026 16:29:20 GMT
• Title: Align, Unify, Suppress, Route: A Coherentist View of Transformer Computation
• Abstract summary: Coherentist Probabilistic Compositionalism grounds transformer computation in coherentist theories of interpretation.<n>Across 15 models from five architecture families, the suppression, unification, and routing weight-space signatures correlate with held-out activation-level role measures.<n>Explicit contradictions significantly shift a layerwise coherence proxy in 14 models; after removing shared residual covariance, the gap has the predicted direction in every model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Mechanistic interpretability has identified transformer circuits, but lacks a shared vocabulary for describing how their functions compose across tasks and architectures. We introduce Coherentist Probabilistic Compositionalism (CPC), an interpretive framework that grounds transformer computation in coherentist theories of interpretation and describes it through four operator roles. Alignment identifies candidate relations, unification integrates supporting information, suppression reduces incompatible alternatives, and routing carries selected information to the output. Across 15 models from five architecture families, the suppression, unification, and routing weight-space signatures correlate with held-out activation-level role measures above random baselines. Suppression is more stable across tasks than unification. Ablating alignment heads reduces downstream suppressive activity beyond a random-head control in 10 models, but similar effects on no-conflict prompts indicate a general upstream dependency, not contradiction-specific coupling. Explicit contradictions significantly shift a layerwise coherence proxy in 14 models; after removing shared residual covariance, the gap has the predicted direction in every model. Base and instruction-tuned variants preserve induction-head score structure ($r{\geq}0.98$) without a consistent shift of operator signatures towards later layers. These results support CPC as a shared vocabulary for comparing transformer mechanisms while showing that their depth and geometric expression remain architecture-specific.

Related papers

• Interior interpretability with attention rollout: contraction and propagation profiles in Transformers [17.72586636966418]
  We introduce emphinterior interpretability, a propagation-based perspective on internal model organization.<n>We interpret rollout as a row-stochastic operator encoding attention-mediated propagation between feature tokens.
  arXiv  Detail & Related papers  (2026-07-24T14:51:28Z)
• Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning [1.3295383263113112]
  Kan Extension Transformers (KETs) are a unifying categorical framework for a diverse group of Transformer implementations.<n>We include a comprehensive experimental validation of 12 different Transformer implementations varying across strict-causal and predict-detach regimes.
  arXiv  Detail & Related papers  (2026-05-26T16:36:59Z)
• IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory [73.22944697933603]
  Existing pipelines produce opaque, end-to-end outputs that expose no intermediate state for inspection.<n>We present IMPACT-Cycle, a supervisory multi-agent system that reformulates long-video understanding as a multimodal iterative claim-level maintenance.
  arXiv  Detail & Related papers  (2026-04-22T03:03:33Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Engineering Verifiable Modularity in Transformers via Per-Layer Supervision [0.0]
  Ablating an attention head identified as critical for capitalization produces minimal behavioral change because distributed redundancy compensates for damage.<n>We demonstrate that architectural interventions can expose hidden modularity.<n>This es tablishes a methodology for transforming interpretability from passive observation to active control.
  arXiv  Detail & Related papers  (2026-03-08T05:18:14Z)
• OrthoFormer: Instrumental Variable Estimation in Transformer Hidden States via Neural Control Functions [0.0]
  Transformer architectures excel at sequential modeling yet remain fundamentally limited by correlational learning.<n>We propose OrthoFormer, a causally grounded architecture that embeds instrumental variable estimation directly into Transformer blocks via neural control functions.
  arXiv  Detail & Related papers  (2026-03-08T03:05:16Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• To Throw a Stone with Six Birds: On Agents and Agenthood [0.0]
  Six Birds Theory (SBT) treats macroscopic objects as induced closures rather than primitives.<n>We give a type-correct account of agency within SBT.<n>We operationalize this contract in finite controlled systems using four checkable components.
  arXiv  Detail & Related papers  (2026-02-03T10:46:23Z)
• Rethinking Transferable Adversarial Attacks on Point Clouds from a Compact Subspace Perspective [55.919842734983156]
  CoSA is a transferable attack framework that operates within a shared low-dimensional semantic space.<n>CoSA consistently outperforms state-of-the-art transferable attacks.
  arXiv  Detail & Related papers  (2026-01-30T15:48:11Z)
• Structural Entropy Guided Probabilistic Coding [52.01765333755793]
  We propose a novel structural entropy-guided probabilistic coding model, named SEPC.<n>We incorporate the relationship between latent variables into the optimization by proposing a structural entropy regularization loss.<n> Experimental results across 12 natural language understanding tasks, including both classification and regression tasks, demonstrate the superior performance of SEPC.
  arXiv  Detail & Related papers  (2024-12-12T00:37:53Z)
• Do Generative Models Know Disentanglement? Contrastive Learning is All You Need [59.033559925639075]
  We propose an unsupervised and model-agnostic method: Disentanglement via Contrast (DisCo) in the Variation Space. DisCo achieves the state-of-the-art disentanglement given pretrained non-disentangled generative models, including GAN, VAE, and Flow.
  arXiv  Detail & Related papers  (2021-02-21T08:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.