A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations

• URL: http://arxiv.org/abs/2603.00824v1
• Date: Sat, 28 Feb 2026 22:05:14 GMT
• Title: A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations
• Abstract summary: We develop a framework for superposition in large language models (LLMs) that replaces the single-global-dictionary premise with a sheaf-theoretic atlas of local semantic charts.<n> Contexts are clustered into a stratified context complex; each chart carries a local feature space and a local information-geometric metric.<n>This yields a Fisher-weighted interference energy and three measurable obstructions to global interpretability.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop a discrete gauge-theoretic framework for superposition in large language models (LLMs) that replaces the single-global-dictionary premise with a sheaf-theoretic atlas of local semantic charts. Contexts are clustered into a stratified context complex; each chart carries a local feature space and a local information-geometric metric (Fisher/Gauss--Newton) identifying predictively consequential feature interactions. This yields a Fisher-weighted interference energy and three measurable obstructions to global interpretability: (O1) local jamming (active load exceeds Fisher bandwidth), (O2) proxy shearing (mismatch between geometric transport and a fixed correspondence proxy), and (O3) nontrivial holonomy (path-dependent transport around loops). We prove and instantiate four results on a frozen open LLM (Llama~3.2~3B Instruct) using WikiText-103, a C4-derived English web-text subset, and \texttt{the-stack-smol}. (A) After constructive gauge fixing on a spanning tree, each chord residual equals the holonomy of its fundamental cycle, making holonomy computable and gauge-invariant. (B) Shearing lower-bounds a data-dependent transfer mismatch energy, turning $D_{\mathrm{shear}}$ into an unavoidable failure bound. (C) We obtain non-vacuous certified jamming/interference bounds with high coverage and zero violations across seeds/hyperparameters. (D) Bootstrap and sample-size experiments show stable estimation of $D_{\mathrm{shear}}$ and $D_{\mathrm{hol}}$, with improved concentration on well-conditioned subsystems.

Related papers

• Mapping and Measuring the Behavioral Evolution of Large Language Models [28.603443859513153]
  Benchmark leaderboards summarize how well a language model performs, but not how its behavior relates to that of other models or changes across generations.<n>We characterize the output behavior of 32 models from six families using their responses to a shared bank of 10,000 prompts.
  arXiv  Detail & Related papers  (2026-08-11T15:07:28Z)
• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• Retrieval-Warmed Energy-Based Reasoning: A Five-Arm Ablation Methodology for Diffusion-as-Inference on Structured Reasoning Tasks [7.954260884574662]
  textbfwarmretrieval-ed energy-based reasoning (RW-EBR) -- an IRED energy-based diffusion model.<n>We study textbfwarmretrieval-ed energy-based reasoning (RW-EBR) -- an IRED energy-based diffusion model.<n>We study textbfwarmretrieval-ed energy-based reasoning (RW-EBR) -- an IRED energy-based diffusion model.
  arXiv  Detail & Related papers  (2026-06-25T00:28:08Z)
• CasualSynth: Generating Structurally Sound Synthetic Data [44.80087038178069]
  Large Language Models (LLMs) generate realistic synthetic data but offer no guarantee that their outputs respect the causal mechanisms governing the target domain.<n>We introduce Causal Synth, a framework that decouples causal structure generation from semantic realization, yielding synthetic data that is both causally valid and linguistically rich.
  arXiv  Detail & Related papers  (2026-05-17T16:21:01Z)
• Entropic Riemannian Neural Optimal Transport [41.69130102668252]
  We propose a unified framework that combines intrinsic entropic OT and amortized out-of-sample evaluation.<n>Our method matches or improves over Euclidean, tangent-space, and log-Euclidean baselines on benchmarks over $mathbbS2$, $mathrmSO(3)$, $mathrmSPD(3)$, $mathrmSE(3)$, and $mathbbH2$.
  arXiv  Detail & Related papers  (2026-05-05T19:43:03Z)
• Entropy-Controlled Flow Matching [0.08460698440162889]
  We propose a constrained variational principle over continuity-equation paths enforcing a global entropy-rate budget d/dt H(mu_t) >= -lambda.<n>We obtain certificate-style mode-coverage and density-floor guarantees with Lipschitz, and construct near-optimal counterexamples for unconstrained flow matching.
  arXiv  Detail & Related papers  (2026-02-25T06:07:01Z)
• Mean-Field Path-Integral Diffusion: From Samples to Interacting Agents [0.0]
  Mean-Field Path-Integral Diffusion (MF-PID) is a framework in which samples are promoted to interacting agents whose drift depends self-consistently on the evolving population density.<n>MF-PID achieves 19--24% reductions in cumulative control energy over independent-agent baselines while matching the prescribed terminal distribution exactly.
  arXiv  Detail & Related papers  (2026-02-23T21:14:06Z)
• Bell-like States in Classical Optics: A Process-Theoretic and Sheaf-Theoretic (Categorical) Clarification [0.0]
  Classical polarization optics is naturally described by a two-dimensional complex Hilbert space (Jones) vectors.<n>For statistical (stochastic) optical fields, suitably prepared two-beam polarization states can exhibit contextuality correlations of quantum strength.<n>The same platform offers a tunable, low-cost testbed for stress-testing Bell/CHSH and contextuality witnesses under realistic imperfections.
  arXiv  Detail & Related papers  (2026-02-16T06:40:05Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• Riemannian Flow Matching for Disentangled Graph Domain Adaptation [51.98961391065951]
  Graph Domain Adaptation (GDA) typically uses adversarial learning to align graph embeddings in Euclidean space.<n>DisRFM is a geometry-aware GDA framework that unifies embedding and flow-based transport.
  arXiv  Detail & Related papers  (2026-01-31T11:05:35Z)
• Latent Object Permanence: Topological Phase Transitions, Free-Energy Principles, and Renormalization Group Flows in Deep Transformer Manifolds [0.5729426778193398]
  We study the emergence of multi-step reasoning in deep Transformer language models through a geometric and statistical-physics lens.<n>We formalize the forward pass as a discrete coarse-graining map and relate the appearance of stable "concept basins" to fixed points of this renormalization-like dynamics.<n>The resulting low-entropy regime is characterized by a spectral tail collapse and by the formation of transient, reusable object-like structures in representation space.
  arXiv  Detail & Related papers  (2026-01-16T23:11:02Z)
• HypoSpace: Evaluating LLM Creativity as Set-Valued Hypothesis Generators under Underdetermination [46.896452542901805]
  We introduce HypoSpace, a diagnostic suite that treats LLMs as samplers of finite hypothesis sets.<n>We instantiate HypoSpace in three structured domains with deterministic validators and exactly enumerated hypothesis spaces.<n>Across instruction-tuned and reasoning-focused models, Validity often remains high while Uniqueness and Recovery degrade as the admissible space grows.
  arXiv  Detail & Related papers  (2025-10-17T13:00:32Z)
• Rectified Diffusion Guidance for Conditional Generation [94.83538269086613]
  We revisit the theory behind CFG and rigorously confirm that the improper combination coefficients (textiti.e.) brings about expectation shift the generative distribution.<n>We show that our approach enjoys a textbftextitform solution given the strength.<n> Empirical evidence on real-world data demonstrate the compatibility of our design with existing state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2024-10-24T13:41:32Z)
• Closed-Loop Unsupervised Representation Disentanglement with $β$-VAE Distillation and Diffusion Probabilistic Feedback [26.935057415769876]
  Representation disentanglement may help AI fundamentally understand the real world and thus benefit both discrimination and generation tasks.<n>We propose a textbfCL-Disentanglement approach dubbed textbfCL-Dis.<n>Experiments demonstrate the superiority of CL-Dis on applications like real image manipulation and visual analysis.
  arXiv  Detail & Related papers  (2024-02-04T05:03:22Z)
• Quantum Darwinism-encoding transitions on expanding trees [0.0]
  We show that quantum dynamics interpolating between broadcasting and scrambling may display sharp phase transitions of information propagation. We find three phases: QD, intermediate and encoding, and two continuous transitions.
  arXiv  Detail & Related papers  (2023-12-07T13:08:11Z)
• Bayesian Renormalization [68.8204255655161]
  We present a fully information theoretic approach to renormalization inspired by Bayesian statistical inference. The main insight of Bayesian Renormalization is that the Fisher metric defines a correlation length that plays the role of an emergent RG scale. We provide insight into how the Bayesian Renormalization scheme relates to existing methods for data compression and data generation.
  arXiv  Detail & Related papers  (2023-05-17T18:00:28Z)
• Averaging Spatio-temporal Signals using Optimal Transport and Soft Alignments [110.79706180350507]
  We show that our proposed loss can be used to define temporal-temporal baryechecenters as Fr'teche means duality. Experiments on handwritten letters and brain imaging data confirm our theoretical findings.
  arXiv  Detail & Related papers  (2022-03-11T09:46:22Z)
• Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling [67.75095378830694]
  We develop a two-stage odometry estimation network, where we obtain the ego-motion by estimating a set of sub-region transformations. In this paper, we aim to alleviate the influence of unreliable structures in training, inference and mapping phases. Our two-frame odometry outperforms the previous state of the arts by 16%/12% in terms of translational/rotational errors.
  arXiv  Detail & Related papers  (2022-02-27T12:52:27Z)
• Unifying supervised learning and VAEs -- coverage, systematics and goodness-of-fit in normalizing-flow based neural network models for astro-particle reconstructions [0.0]
  Statistical uncertainties, coverage, systematic uncertainties or a goodness-of-fit measure are often not calculated. We show that a KL-divergence objective of the joint distribution of data and labels allows to unify supervised learning and variational autoencoders. We discuss how to calculate coverage probabilities without numerical integration for specific "base-ordered" contours.
  arXiv  Detail & Related papers  (2020-08-13T11:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.