Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signal

• URL: http://arxiv.org/abs/2602.17532v1
• Date: Thu, 19 Feb 2026 16:43:12 GMT
• Title: Systematic Evaluation of Single-Cell Foundation Model Interpretability Reveals Attention Captures Co-Expression Rather Than Unique Regulatory Signal
• Abstract summary: We present a framework for assessing mechanistic interpretability in single-cell foundation models.<n>Applying this framework to scGPT and Geneformer, we find that attention patterns encode structured biological information with layer-specific organisation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a systematic evaluation framework - thirty-seven analyses, 153 statistical tests, four cell types, two perturbation modalities - for assessing mechanistic interpretability in single-cell foundation models. Applying this framework to scGPT and Geneformer, we find that attention patterns encode structured biological information with layer-specific organisation - protein-protein interactions in early layers, transcriptional regulation in late layers - but this structure provides no incremental value for perturbation prediction: trivial gene-level baselines outperform both attention and correlation edges (AUROC 0.81-0.88 versus 0.70), pairwise edge scores add zero predictive contribution, and causal ablation of regulatory heads produces no degradation. These findings generalise from K562 to RPE1 cells; the attention-correlation relationship is context-dependent, but gene-level dominance is universal. Cell-State Stratified Interpretability (CSSI) addresses an attention-specific scaling failure, improving GRN recovery up to 1.85x. The framework establishes reusable quality-control standards for the field.

Related papers

• BioM-JEPA: joint-embedding prediction of graph-connected gene blocks in single cells [64.21963170814706]
  BioM-JEPA predicts aggregate representations of graph-connected gene blocks.<n>A student network infers each target-block representation from the remaining genes in a cell.<n>Across CellBench tasks, BioM-JEPA embeddings retained expression, pathway and neighbourhood information.
  arXiv  Detail & Related papers  (2026-08-06T11:58:29Z)
• CoDiffGRN: Rethinking Gene Regulatory Network Inference via the BEELINE-KGC Benchmark and Co-evolutionary Discrete Diffusion [20.970593459575756]
  Inferring gene regulatory networks (GRNs) from single-cell transcriptomic data is crucial for biological discovery.<n>We reformulate GRN inference as an inductive, ranking-centric graph completion problem.<n>We introduce textbfbenchmark, a new benchmark that incorporates an inductive gene-holdout split together with knowledge graph completion metrics.
  arXiv  Detail & Related papers  (2026-07-14T16:07:23Z)
• Structure-Aware Prediction of PROTAC-Mediated Protein Degradability via Graph Neural Networks [0.0]
  We present DegradoMap, a graph neural network that predicts PROTAC-mediated degradability from protein structure and E3 ligase identity alone.<n>The model encodes biophysical priors through lysine-weighted graph pooling with per-protein normalization.<n>It achieves 0.646+-0.124 AUROC on target-unseen evaluation and 0.811 AUROC on CRBN->VHL E3-unseen transfer.
  arXiv  Detail & Related papers  (2026-06-01T09:39:22Z)
• STaR-DRO: Stateful Tsallis Reweighting for Group-Robust Structured Prediction [0.20165644958637055]
  We present a two-part framework for controllable inference and robust fine-tuning.<n>First, we introduce a task-agnostic prompting strategy to address format drift, label ambiguity, evidence hallucination, and metadata-conditioned confusion.<n>Second, we introduce STaR-DRO, a stateful robust optimization method for group heterogeneity.
  arXiv  Detail & Related papers  (2026-04-09T21:58:42Z)
• Quantum-Inspired Geometric Classification with Correlation Group Structures and VQC Decision Modeling [0.5039813366558307]
  We propose a geometry-driven quantum-inspired classification framework that integrates Correlation Group Structures (CGR), compact SWAP-test-based overlap estimation, and selective variational quantum decision modelling.<n>CGR organizes features into anchor-centered correlation neighbourhoods, generating nonlinear, correlation-weighted representations.<n>On the Credit Card Fraud dataset (0.17% prevalence), the Delta + VQC pipeline achieves approximately 0.85 minority recall at an alert rate of approximately 1.31%, with ROC-AUC 0.9249 and PR-AUC 0.3251 under full-dataset evaluation.
  arXiv  Detail & Related papers  (2026-04-02T11:50:29Z)
• GOLDMARK: Governed Outcome-Linked Diagnostic Model Assessment Reference Kit [7.891604152731754]
  Computational biomarkers (CBs) are histo-derived patterns extracted from hematoxylin-eosin (H&E) whole-slide images (WSIs) to predict therapeutic response or prognosis.<n>We introduce GOLDMARK, a standardized benchmarking framework built on a curated TCGA cohort with clinically actionable OncoKB level 1-3 biomarker labels.
  arXiv  Detail & Related papers  (2026-03-21T15:09:06Z)
• Beyond Independent Genes: Learning Module-Inductive Representations for Gene Perturbation Prediction [48.80217316452559]
  scBIG is a module-inductive prediction framework that explicitly models coordinated gene programs.<n> scBIG consistently outperforms state-of-the-art methods, particularly on unseen and perturbation settings.
  arXiv  Detail & Related papers  (2026-02-03T16:43:40Z)
• Automated Identification of Incidentalomas Requiring Follow-Up: A Multi-Anatomy Evaluation of LLM-Based and Supervised Approaches [5.958100741754613]
  We evaluated large language models (LLMs) against supervised baselines for fine-grained, lesion-level detection of incidentalomas.<n>We introduced a novel inference strategy using lesion-tagged inputs and anatomy-aware prompting to ground model reasoning.<n>The anatomy-informed GPT-OSS-20b model achieved the highest performance, yielding an incidentaloma-positive macro-F1 of 0.79.
  arXiv  Detail & Related papers  (2025-12-05T08:49:57Z)
• CONFIDE: Hallucination Assessment for Reliable Biomolecular Structure Prediction and Design [46.12506067241116]
  We present CODE (Chain of Diffusion Embeddings), a self evaluating metric to quantify topological frustration.<n>We propose CONFIDE, a unified evaluation framework that combines energetic and topological perspectives.<n>By combining data driven embeddings with theoretical insight, CODE and CONFIDE outperform existing metrics across a wide range of biomolecular systems.
  arXiv  Detail & Related papers  (2025-11-20T03:38:46Z)
• Soft Graph Clustering for single-cell RNA Sequencing Data [15.648907695773701]
  We introduce scSGC, a Soft Graph Clustering for single-cell RNA sequencing data.<n> scSGC aims to more accurately characterize continuous similarities among cells through non-binary edge weights.<n>It outperforms 13 state-of-the-art clustering models in clustering accuracy, cell type annotation, and computational efficiency.
  arXiv  Detail & Related papers  (2025-07-14T03:49:12Z)
• DISPROTBENCH: A Disorder-Aware, Task-Rich Benchmark for Evaluating Protein Structure Prediction in Realistic Biological Contexts [76.59606029593085]
  DisProtBench is a benchmark for evaluating protein structure prediction models (PSPMs) under structural disorder and complex biological conditions.<n>DisProtBench spans three key axes: data complexity, task diversity, and Interpretability.<n>Results reveal significant variability in model robustness under disorder, with low-confidence regions linked to functional prediction failures.
  arXiv  Detail & Related papers  (2025-06-18T23:58:22Z)
• GRNFormer: A Biologically-Guided Framework for Integrating Gene Regulatory Networks into RNA Foundation Models [39.58414436685698]
  We propose a new framework that integrates multi-scale Gene Regulatory Networks (GRNs) inferred from multi-omics data into RNA foundation model training.<n>GRNFormer achieves consistent improvements over state-of-the-art (SoTA) baselines.
  arXiv  Detail & Related papers  (2025-03-03T15:56:39Z)
• CRTRE: Causal Rule Generation with Target Trial Emulation Framework [47.2836994469923]
  We introduce a novel method called causal rule generation with target trial emulation framework (CRTRE) CRTRE applies randomize trial design principles to estimate the causal effect of association rules. We then incorporate such association rules for the downstream applications such as prediction of disease onsets.
  arXiv  Detail & Related papers  (2024-11-10T02:40:06Z)
• Single-Cell Deep Clustering Method Assisted by Exogenous Gene Information: A Novel Approach to Identifying Cell Types [50.55583697209676]
  We develop an attention-enhanced graph autoencoder, which is designed to efficiently capture the topological features between cells. During the clustering process, we integrated both sets of information and reconstructed the features of both cells and genes to generate a discriminative representation. This research offers enhanced insights into the characteristics and distribution of cells, thereby laying the groundwork for early diagnosis and treatment of diseases.
  arXiv  Detail & Related papers  (2023-11-28T09:14:55Z)
• Learning to diagnose cirrhosis from radiological and histological labels with joint self and weakly-supervised pretraining strategies [62.840338941861134]
  We propose to leverage transfer learning from large datasets annotated by radiologists, to predict the histological score available on a small annex dataset. We compare different pretraining methods, namely weakly-supervised and self-supervised ones, to improve the prediction of the cirrhosis. This method outperforms the baseline classification of the METAVIR score, reaching an AUC of 0.84 and a balanced accuracy of 0.75.
  arXiv  Detail & Related papers  (2023-02-16T17:06:23Z)
• Granger causal inference on DAGs identifies genomic loci regulating transcription [77.58911272503771]
  GrID-Net is a framework based on graph neural networks with lagged message passing for Granger causal inference on DAG-structured systems. Our application is the analysis of single-cell multimodal data to identify genomic loci that mediate the regulation of specific genes.
  arXiv  Detail & Related papers  (2022-10-18T21:15:10Z)
• An Integrated Deep Learning and Dynamic Programming Method for Predicting Tumor Suppressor Genes, Oncogenes, and Fusion from PDB Structures [0.0]
  Mutations in proto-oncogenes (ONGO) and the loss of regulatory function of tumor suppression genes (TSG) are the common underlying mechanism for uncontrolled tumor growth. Finding the potentiality of the genes related functionality to ONGO or TSG through computational studies can help develop drugs that target the disease. This paper proposes a classification method that starts with a preprocessing stage to extract the feature map sets from the input 3D protein structural information.
  arXiv  Detail & Related papers  (2021-05-17T18:18:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.