Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control

• URL: http://arxiv.org/abs/2603.11940v1
• Date: Thu, 12 Mar 2026 13:53:20 GMT
• Title: Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control
• Abstract summary: We present three experiments that address limitations through exhaustive circuit tracing, higher order ablation, and causal trajectory steering.<n>First, exhaustive tracing of all 4065 active autoencoder features at layer 5 yields 1393850 significant downstream edges.<n>Second, three way redundancy across 8 feature triplets shows that redundancy deepens monotonically with interaction order.<n>Third, trajectory guided feature steering establishes a causal link between layer position and differentiation directionality.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mechanistic interpretability of biological foundation models has relied on selective feature sampling, pairwise interaction testing, and observational trajectory analysis. Each of these can introduce systematic bias. Here we present three experiments that address these limitations through exhaustive circuit tracing, higher order combinatorial ablation, and causal trajectory steering in Geneformer, a transformer based single cell foundation model. First, exhaustive tracing of all 4065 active sparse autoencoder features at layer 5 yields 1393850 significant downstream edges, a 27 fold expansion over selective sampling. This reveals a heavy tailed hub distribution in which 1.8 percent of features account for disproportionate connectivity and 40 percent of the top 20 hubs lack biological annotation. These results indicate systematic annotation bias in prior selective analyses. Second, three way combinatorial ablation across 8 feature triplets shows that redundancy deepens monotonically with interaction order, with a three way ratio of 0.59 versus a pairwise ratio of 0.74, and with zero synergy. This confirms that the model architecture is subadditive at all tested orders. Third, trajectory guided feature steering establishes a causal link between layer position and differentiation directionality. Late layer features at L17 consistently push cell states toward maturity, with fraction positive equal to 1.0. Early and mid layer features at L0 and L11 mostly push away from maturity, with fraction positive ranging from 0.00 to 0.58. Together these results move from correlation toward causal evidence for layer dependent control of cell state.

Related papers

• From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts [61.695980784491475]
  Test-time scaling produces cohorts of reasoning rollouts, yet there is no standard label-free account of how their internal computation reorganizes as inference unfolds.<n>We introduce routing effective rank deff, the entropy-effective dimensionality of a cross-rollout graph built from MoE expert-routing similarity.
  arXiv  Detail & Related papers  (2026-09-06T05:43:22Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Orientation, not magnitude: the causal structure of task-vector interference in merged language models [0.0]
  We show that task arithmetic works until it doesn't, and the field diagnoses why with magnitudes.<n>An exact decomposition of the layerwise flux shows it is dominated by amplifying transport of the existing cross-term.<n>The striking +-15% "universality" of naive bfloat16 generation turns out to be quantization roughness.
  arXiv  Detail & Related papers  (2026-08-12T08:40:24Z)
• From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models [0.0]
  We propose a five-stage methodology for causal feature analysis in transformer language models.<n>We demonstrate it end-to-end on GPT-2 small performing the Indirect Object Identification (IOI) task.
  arXiv  Detail & Related papers  (2026-05-21T13:25:16Z)
• Shared Emotion Geometry Across Small Language Models: A Cross-Architecture Study of Representation, Behavior, and Methodological Confounds [4.358468367889626]
  We extract 21-emotion vector sets from twelve small language models under a unified comprehension-mode pipeline at fp16 precision.<n>The five mature architectures share nearly identical 21-emotion geometry, with pairwise RDM Spearman correlations of 0.74-0.92.<n>We show that what prior work has read as a single comprehension-vs.generation method effect in fact decomposes into four distinct layers.
  arXiv  Detail & Related papers  (2026-04-13T06:27:40Z)
• The Coordinate System Problem in Persistent Structural Memory for Neural Architectures [0.0]
  We introduce the Dual-View Pheromone Pathway Network (DPPN), an architecture that routes sparse attention through a persistent pheromone field over latent slot transitions.<n>We identify a core principle: persistent memory requires a stable coordinate system.<n>We show that neither contrastive updates, multi-source distillation, Hungarian alignment, nor semantic decomposition resolves the instability when embeddings are learned from scratch.
  arXiv  Detail & Related papers  (2026-03-24T06:51:59Z)
• Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence [0.0]
  We introduce causal circuit tracing by ablating SAE features and downstream responses.<n>We apply it to Geneformer V2-316M and scGPT whole-human across four conditions.
  arXiv  Detail & Related papers  (2026-03-02T11:21:44Z)
• What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses [0.0]
  We propose an AI-driven-brainstormer loop that proposed, tested, and refined 141 geometric and topological hypotheses across 52 iterations.<n>Gene embedding neighborhoods exhibit non-trivial topology, with persistent homology significant in 11 of 12 transformer layers.<n> CCA alignment between scGPT and Geneformer yields canonical correlation of 0.80 and gene retrieval accuracy of 72 percent, yet none of 19 tested methods reliably recover gene-level correspondences.
  arXiv  Detail & Related papers  (2026-02-25T14:33:24Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Structure-Informed Estimation for Pilot-Limited MIMO Channels via Tensor Decomposition [51.56484100374058]
  This paper formulates pilot-limited channel estimation as low-rank tensor completion from sparse observations.<n>Experiments on synthetic channels demonstrate 10-20,dB normalized mean-square error (NMSE) improvement over least-squares (LS)<n> evaluations on DeepMIMO ray-tracing channels show 24-44% additional NMSE reduction over pure tensor-based methods.
  arXiv  Detail & Related papers  (2026-02-03T23:38:05Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Beyond Mimicry: Preference Coherence in LLMs [0.19116784879310025]
  We investigate whether large language models exhibit genuine preference structures by testing their responses to AI-specific trade-offs.<n>We find 23 combinations (47.9%) demonstrated statistically significant relationships between scenario intensity and choice patterns.<n>Only 5 combinations (10.4%) demonstrate meaningful preference coherence through adaptive or threshold-based behavior.<n>The prevalence of unstable transitions (45.8%) and stimulus-specific sensitivities suggests current AI systems lack unified preference structures.
  arXiv  Detail & Related papers  (2025-11-17T17:41:48Z)
• Departures: Distributional Transport for Single-Cell Perturbation Prediction with Neural Schrödinger Bridges [51.83259180910313]
  A major bottleneck in gene function analysis is the unpaired nature of single-cell data.<n>We approximate Schrdinger Bridge (SB) to tackle unpaired single-cell perturbation data.<n>Our model effectively captures heterogeneous single-cell responses and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-11-17T08:27:13Z)
• Balancing Centralized Learning and Distributed Self-Organization: A Hybrid Model for Embodied Morphogenesis [0.0]
  We investigate how to couple a learnable brain-like'' controller to a cell-like'' Gray--Scott substrate to steer pattern formation with minimal effort.<n>A compact convolutional policy is embedded in a differentiable PyTorch reaction--diffusion simulator.
  arXiv  Detail & Related papers  (2025-11-13T09:05:27Z)
• Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail [85.47497935739936]
  Alpamayo-R1 (AR1) is a vision-language-action model that integrates Chain of Causation reasoning with trajectory planning.<n>We show AR1 achieves 12% improvement in planning accuracy on challenging cases compared to a trajectory-only baseline.<n>We plan to release AR1 models and a subset of the CoC in a future update.
  arXiv  Detail & Related papers  (2025-10-30T01:25:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.