Discovering Interpretable Biological Concepts in Single-cell RNA-seq Foundation Models

• URL: http://arxiv.org/abs/2510.25807v1
• Date: Wed, 29 Oct 2025 08:52:55 GMT
• Title: Discovering Interpretable Biological Concepts in Single-cell RNA-seq Foundation Models
• Authors: Charlotte Claye, Pierre Marschall, Wassila Ouerdane, Céline Hudelot, Julien Duquesne,
• Abstract summary: Single-cell RNA-seq foundation models achieve strong performance on downstream tasks but remain black boxes.<n>Recent work has shown that sparse dictionary learning can extract concepts from deep learning models.<n>We introduce a novel concept-based interpretability framework for single-cell RNA-seq models.
• Score: 3.810388351528255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single-cell RNA-seq foundation models achieve strong performance on downstream tasks but remain black boxes, limiting their utility for biological discovery. Recent work has shown that sparse dictionary learning can extract concepts from deep learning models, with promising applications in biomedical imaging and protein models. However, interpreting biological concepts remains challenging, as biological sequences are not inherently human-interpretable. We introduce a novel concept-based interpretability framework for single-cell RNA-seq models with a focus on concept interpretation and evaluation. We propose an attribution method with counterfactual perturbations that identifies genes that influence concept activation, moving beyond correlational approaches like differential expression analysis. We then provide two complementary interpretation approaches: an expert-driven analysis facilitated by an interactive interface and an ontology-driven method with attribution-based biological pathway enrichment. Applying our framework to two well-known single-cell RNA-seq models from the literature, we interpret concepts extracted by Top-K Sparse Auto-Encoders trained on two immune cell datasets. With a domain expert in immunology, we show that concepts improve interpretability compared to individual neurons while preserving the richness and informativeness of the latent representations. This work provides a principled framework for interpreting what biological knowledge foundation models have encoded, paving the way for their use for hypothesis generation and discovery.

Related papers

• SC-Arena: A Natural Language Benchmark for Single-Cell Reasoning with Knowledge-Augmented Evaluation [24.956743572453153]
  We present SC-ARENA, a natural language evaluation framework tailored to single-cell foundation models.<n>SC-ARENA formalizes a virtual cell abstraction that unifies evaluation targets by representing both intrinsic attributes and gene-level interactions.
  arXiv  Detail & Related papers  (2026-02-26T16:50:28Z)
• Concept-Guided Interpretability via Neural Chunking [64.6429903327095]
  We show that neural networks exhibit patterns in their raw population activity that mirror regularities in the training data.<n>We propose three methods to extract recurring chunks on a neural population level.<n>Our work points to a new direction for interpretability, one that harnesses both cognitive principles and the structure of naturalistic data.
  arXiv  Detail & Related papers  (2025-05-16T13:49:43Z)
• Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification [55.98854157265578]
  Life-Code is a comprehensive framework that spans different biological functions.<n>We propose a unified pipeline to integrate multi-omics data by reverse-transcribing RNA and reverse-translating amino acids into nucleotide-based sequences.<n>Life-Code achieves state-of-the-art results on various tasks across three omics, highlighting its potential for advancing multi-omics analysis and interpretation.
  arXiv  Detail & Related papers  (2025-02-11T06:53:59Z)
• Neural DNF-MT: A Neuro-symbolic Approach for Learning Interpretable and Editable Policies [51.03989561425833]
  We propose a neuro-symbolic approach called neural DNF-MT for end-to-end policy learning.<n>The differentiable nature of the neural DNF-MT model enables the use of deep actor-critic algorithms for training.<n>We show how the bivalent representations of deterministic policies can be edited and incorporated back into a neural model.
  arXiv  Detail & Related papers  (2025-01-07T15:51:49Z)
• Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models [6.136186137141521]
  We propose a novel combination of a sparse dictionary learning algorithm, Iterative Codebook Feature Learning (ICFL), with a PCA whitening pre-processing step derived from control data.<n>We successfully retrieve biologically meaningful concepts, such as cell types and genetic perturbations.<n>Our method offers a promising new direction for scientific discovery via mechanistic interpretability in bioimaging.
  arXiv  Detail & Related papers  (2024-12-20T00:01:16Z)
• Causal Representation Learning from Multimodal Biomedical Observations [57.00712157758845]
  We develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets.<n>Key theoretical contribution is the structural sparsity of causal connections between modalities.<n>Results on a real-world human phenotype dataset are consistent with established biomedical research.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Learning biologically relevant features in a pathology foundation model using sparse autoencoders [2.5919097694815365]
  We trained a Sparse Autoencoder on the embeddings of a pathology pretrained foundation model.<n>We found that Sparse Autoencoder features represent interpretable and monosemantic biological concepts.
  arXiv  Detail & Related papers  (2024-07-15T15:03:01Z)
• Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields [56.69755544814834]
  We present evidence suggesting that depthwise convolutional kernels are effectively replicating the biological receptive fields observed in the mammalian retina. We propose a scheme that draws inspiration from the biological receptive fields.
  arXiv  Detail & Related papers  (2024-01-18T18:06:22Z)
• Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
  Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation. We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
  arXiv  Detail & Related papers  (2023-11-08T20:41:18Z)
• A Recursive Bateson-Inspired Model for the Generation of Semantic Formal Concepts from Spatial Sensory Data [77.34726150561087]
  This paper presents a new symbolic-only method for the generation of hierarchical concept structures from complex sensory data. The approach is based on Bateson's notion of difference as the key to the genesis of an idea or a concept. The model is able to produce fairly rich yet human-readable conceptual representations without training.
  arXiv  Detail & Related papers  (2023-07-16T15:59:13Z)
• Biologically-informed deep learning models for cancer: fundamental trends for encoding and interpreting oncology data [0.0]
  We provide a structured literature analysis focused on Deep Learning (DL) models used to support inference in cancer biology. The work focuses on how existing models address the need for better dialogue with prior knowledge, biological plausibility and interpretability.
  arXiv  Detail & Related papers  (2022-07-02T12:11:35Z)
• Evolution Is All You Need: Phylogenetic Augmentation for Contrastive Learning [1.7188280334580197]
  Self-supervised representation learning of biological sequence embeddings alleviates computational resource constraints on downstream tasks. We show that contrastive learning using evolutionary phylogenetic augmentation can be used as a representation learning objective.
  arXiv  Detail & Related papers  (2020-12-25T01:35:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.