Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence

• URL: http://arxiv.org/abs/2603.01752v1
• Date: Mon, 02 Mar 2026 11:21:44 GMT
• Title: Causal Circuit Tracing Reveals Distinct Computational Architectures in Single-Cell Foundation Models: Inhibitory Dominance, Biological Coherence, and Cross-Model Convergence
• Authors: Ihor Kendiukhov,
• Abstract summary: We introduce causal circuit tracing by ablating SAE features and downstream responses.<n>We apply it to Geneformer V2-316M and scGPT whole-human across four conditions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Motivation: Sparse autoencoders (SAEs) decompose foundation model activations into interpretable features, but causal feature-to-feature interactions across network depth remain unknown for biological foundation models. Results: We introduce causal circuit tracing by ablating SAE features and measuring downstream responses, and apply it to Geneformer V2-316M and scGPT whole-human across four conditions (96,892 edges, 80,191 forward passes). Both models show approximately 53 percent biological coherence and 65 to 89 percent inhibitory dominance, invariant to architecture and cell type. scGPT produces stronger effects (mean absolute d = 1.40 vs. 1.05) with more balanced dynamics. Cross-model consensus yields 1,142 conserved domain pairs (10.6x enrichment, p < 0.001). Disease-associated domains are 3.59x more likely to be consensus. Gene-level CRISPRi validation shows 56.4 percent directional accuracy, confirming co-expression rather than causal encoding.

Related papers

• Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single-cell foundation models: a comparative atlas of Geneformer and scGPT [0.0]
  Single-cell foundation models Geneformer and scGPT encode rich biological information.<n>We trained TopK SAEs on residual stream activations from all layers of Geneformer V2-316M and scGPT whole-human.<n>We release both feature atlases as interactive web platforms enabling exploration of more than 107000 features across 30 layers of two leading single-cell foundation models.
  arXiv  Detail & Related papers  (2026-03-03T13:05:11Z)
• What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses [0.0]
  We propose an AI-driven-brainstormer loop that proposed, tested, and refined 141 geometric and topological hypotheses across 52 iterations.<n>Gene embedding neighborhoods exhibit non-trivial topology, with persistent homology significant in 11 of 12 transformer layers.<n> CCA alignment between scGPT and Geneformer yields canonical correlation of 0.80 and gene retrieval accuracy of 72 percent, yet none of 19 tested methods reliably recover gene-level correspondences.
  arXiv  Detail & Related papers  (2026-02-25T14:33:24Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Central Dogma Transformer: Towards Mechanism-Oriented AI for Cellular Understanding [0.0]
  We present the Central Dogma Transformer (CDT), an architecture that integrates pre-trained language models for DNA, RNA, and protein.<n>We validate CDT v1 on CRISPRi enhancer perturbation data from K562 cells, achieving a Pearson correlation of 0.503.<n>These results suggest that AI architectures aligned with biological information flow can achieve both predictive accuracy and mechanistic interpretability.
  arXiv  Detail & Related papers  (2026-01-03T06:29:22Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• CONFIDE: Hallucination Assessment for Reliable Biomolecular Structure Prediction and Design [46.12506067241116]
  We present CODE (Chain of Diffusion Embeddings), a self evaluating metric to quantify topological frustration.<n>We propose CONFIDE, a unified evaluation framework that combines energetic and topological perspectives.<n>By combining data driven embeddings with theoretical insight, CODE and CONFIDE outperform existing metrics across a wide range of biomolecular systems.
  arXiv  Detail & Related papers  (2025-11-20T03:38:46Z)
• Characterizing Continuous and Discrete Hybrid Latent Spaces for Structural Connectomes [1.7732287904031832]
  Structural connectomes are detailed graphs that map how different brain regions are physically connected.<n>Low-dimensional spaces like PCA and autoencoders are often used to capture major sources of variation.<n>We propose a variational autoencoder (VAE) with a hybrid latent space that jointly models the discrete and continuous components.
  arXiv  Detail & Related papers  (2025-11-20T02:52:17Z)
• UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials [62.72989417755985]
  We present UniGenX, a unified generative model for function in natural systems.<n>UniGenX represents heterogeneous inputs as a mixed stream of symbolic and numeric tokens.<n>It achieves state-of-the-art or competitive performance for the function-aware generation across domains.
  arXiv  Detail & Related papers  (2025-03-09T16:43:07Z)
• CRTRE: Causal Rule Generation with Target Trial Emulation Framework [47.2836994469923]
  We introduce a novel method called causal rule generation with target trial emulation framework (CRTRE) CRTRE applies randomize trial design principles to estimate the causal effect of association rules. We then incorporate such association rules for the downstream applications such as prediction of disease onsets.
  arXiv  Detail & Related papers  (2024-11-10T02:40:06Z)
• CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense [61.78357530675446]
  Humans are difficult to be cheated by subtle manipulations, since we make judgments only based on essential factors.<n>Inspired by this observation, we attempt to model label generation with essential label-causative factors and incorporate label-non-causative factors to assist data generation.<n>For an adversarial example, we aim to discriminate perturbations as non-causative factors and make predictions only based on the label-causative factors.
  arXiv  Detail & Related papers  (2024-10-30T15:06:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.