All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs

• URL: http://arxiv.org/abs/2605.12671v1
• Date: Tue, 12 May 2026 19:21:23 GMT
• Title: All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs
• Abstract summary: We show that a single task can be supported by multiple, structurally distinct circuits or sheaves that are simultaneously faithful, sparse, and complete.<n>We propose a Distributive Circuit Hypothesis and provide a theoretical analysis demonstrating that non-unique, low-overlap circuit explanations arise naturally from high-dimensional superposition.
• Score: 61.02007370456733
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present empirical and theoretical evidence against a central but largely implicit assumption in circuit and sheaf discovery (CSD), which we term the Functional Anisotropy Hypothesis: the idea that functions in large language models (LLMs) are localised to a unique or near-unique internal mechanism. We show that a single LLM task can instead be supported by multiple, structurally distinct circuits or sheaves that are simultaneously faithful, sparse, and complete. To systematically uncover such competing mechanisms, we introduce Overlap-Aware Sheaf Repulsion, a method that augments the CSD objective with an explicit penalty on structural overlap across multiple discovery runs, enabling the discovery of circuits or sheaves with strong task performance but minimal shared structure across a plethora of common CSD benchmarks. We find that this phenomenon becomes increasingly pronounced as the number of discovered sheaves grows and persists robustly across major CSD methods. We further identify an ultra-sparse three-edge sheaf and show that none of its edges is individually indispensable, undermining even weakened notions of canonical or essential components. To explain these findings, we propose a Distributive Dense Circuit Hypothesis and provide a theoretical analysis demonstrating that non-unique, low-overlap circuit explanations arise naturally from high-dimensional superposition under mild assumptions. Together, our results suggest that mechanistic explanations in LLMs are inherently non-canonical and call for a rethinking of how CSD results should be interpreted and evaluated.

Related papers

• CDFM: Towards a General-Purpose Causal Discovery Foundation Model [62.32138962508216]
  Causal discovery is the process of recovering underlying causal structures from observational data.<n>We formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference.<n>CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.
  arXiv  Detail & Related papers  (2026-07-13T12:58:28Z)
• Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing [4.400865943835067]
  Causal representation learning seeks to uncover meaningful latent variables and their corresponding causal structure from observational data.<n>Proving identifiability in CRL is intrinsically difficult, and we address in this work an even more challenging setting: multimodality.<n>We consider multimodal observed data with a latent partially shared structure. Each modality is generated, through non linear mixing functions, from a specific subset of causal latent variables.<n>Our identifiability results, furthermore, apply to the undercomplete scenario where we have, for each modality, more observed than latent variables.
  arXiv  Detail & Related papers  (2026-05-18T21:34:29Z)
• PLUME: Latent Reasoning Based Universal Multimodal Embedding [52.35354073629127]
  Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model.<n>Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings.<n>We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states.
  arXiv  Detail & Related papers  (2026-04-02T14:04:53Z)
• The Observer-Situation Lattice: A Unified Formal Basis for Perspective-Aware Cognition [2.28438857884398]
  We introduce the Observer-Situation Lattice (OSL), a unified mathematical structure that provides a single, coherent semantic space for perspective-aware cognition.<n>OSL is a finite complete lattice where each element represents a unique observer-situation pair, allowing for a principled and scalable approach to belief management.<n>We present two key algorithms that operate on this lattice: (i) Relativized Belief propagation, an incremental update algorithm that efficiently propagates new information, and (ii) Minimal Contradiction Decomposition, a graph-based procedure that identifies and isolates contradiction components.
  arXiv  Detail & Related papers  (2026-03-02T03:15:36Z)
• Provable Low-Frequency Bias of In-Context Learning of Representations [19.066378730056275]
  In-context learning (ICL) enables large language models (LLMs) to acquire new behaviors from the input sequence alone without any parameter updates.<n>Recent studies have shown that ICL can surpass the original meaning learned in pretraining stage through internalizing the structure the data-generating process (DGP) of the prompt into the hidden representations.<n>We present the first rigorous explanation of such phenomena by introducing a unified framework of double convergence.<n>This double convergence process leads to an implicit bias towards smooth (low-frequency) representations, which we prove analytically and verify empirically.
  arXiv  Detail & Related papers  (2025-07-17T21:19:32Z)
• CTRLS: Chain-of-Thought Reasoning via Latent State-Transition [57.51370433303236]
  Chain-of-thought (CoT) reasoning enables large language models to break down complex problems into interpretable intermediate steps.<n>We introduce groundingS, a framework that formulates CoT reasoning as a Markov decision process (MDP) with latent state transitions.<n>We show improvements in reasoning accuracy, diversity, and exploration efficiency across benchmark reasoning tasks.
  arXiv  Detail & Related papers  (2025-07-10T21:32:18Z)
• The Unified Cognitive Consciousness Theory for Language Models: Anchoring Semantics, Thresholds of Activation, and Emergent Reasoning [2.0800882594868293]
  Unified Cognitive Consciousness Theory (UCCT) casts them as vast unconscious pattern repositories.<n>UCCT formalizes this process as Bayesian competition between statistical priors learned in pre-training and context-driven target patterns.<n>We ground the theory in three principles: threshold crossing, modality, and density-distance predictive power.
  arXiv  Detail & Related papers  (2025-06-02T18:12:43Z)
• The Theory of the Unique Latent Pattern: A Formal Epistemic Framework for Structural Singularity in Complex Systems [2.44755919161855]
  This paper introduces the Theory of the Unique Latent Pattern (ULP), a formal framework that redefines the origin of apparent complexity in dynamic systems.<n>Rather than attributing unpredictability to intrinsic randomness or emergent nonlinearity, ULP asserts that every analyzable system is governed by a structurally unique, deterministic generative mechanism.
  arXiv  Detail & Related papers  (2025-05-24T19:52:28Z)
• The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning [56.574829311863446]
  Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs)<n>We demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities.<n>Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL.
  arXiv  Detail & Related papers  (2025-04-07T13:51:06Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.