Discovery of a Hematopoietic Manifold in scGPT Yields a Method for Extracting Performant Algorithms from Biological Foundation Model Internals

• URL: http://arxiv.org/abs/2603.10261v1
• Date: Tue, 10 Mar 2026 22:44:12 GMT
• Title: Discovery of a Hematopoietic Manifold in scGPT Yields a Method for Extracting Performant Algorithms from Biological Foundation Model Internals
• Abstract summary: We show that scGPT internally encodes a compact hematopoietic manifold with significant developmental branch structure.<n>We introduce a general three-stage extraction method consisting of direct operator export from frozen attention weights.<n>Mechanistic interpretability of the compact operator reveals a concentrated four-factor core explaining 66.2% of ablation impact.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We report the discovery and extraction of a compact hematopoietic algorithm from the single-cell foundation model scGPT, to our knowledge the first biologically useful, competitive algorithm extracted from a foundation model via mechanistic interpretability. We show that scGPT internally encodes a compact hematopoietic manifold with significant developmental branch structure, validated on a strict non-overlap Tabula Sapiens external panel and confirmed via frozen-head zero-shot transfer to an independent multi-donor immune panel. To isolate this geometry, we introduce a general three-stage extraction method consisting of direct operator export from frozen attention weights, a lightweight learned adaptor, and a task-specific readout, producing a standalone algorithm without target-dataset retraining. In 88-split donor-holdout benchmarks against scVI, Palantir, DPT, CellTypist, PCA, and raw-expression baselines, the extracted algorithm achieves the strongest pseudotime-depth ordering and leads on key subtype endpoints (CD4/CD8 AUROC 0.867, mono/macro AUROC 0.951). Compared to standard probing of frozen scGPT embeddings with a 3-layer MLP, the extracted head is BH-significantly better on 6/8 classification endpoints while completing a full 12-split evaluation campaign 34.5x faster with approximately 1000x fewer trainable parameters. The exported operator compresses from three pooled attention heads to a single head without statistically significant loss, and further to a rank-64 surrogate. Mechanistic interpretability of the compact operator reveals a concentrated four-factor core explaining 66.2% of ablation impact, with factors resolving into explicit T/lymphoid, B/plasma, granulocytic, and monocyte/macrophage gene programs. A supplementary second-manifold validation (intercellular communication geometry) confirms that the extraction method generalizes beyond hematopoiesis.

Related papers

• BioM-JEPA: joint-embedding prediction of graph-connected gene blocks in single cells [64.21963170814706]
  BioM-JEPA predicts aggregate representations of graph-connected gene blocks.<n>A student network infers each target-block representation from the remaining genes in a cell.<n>Across CellBench tasks, BioM-JEPA embeddings retained expression, pathway and neighbourhood information.
  arXiv  Detail & Related papers  (2026-08-06T11:58:29Z)
• CorePath: A Breast-Specialized Pathology Foundation Model for Core Needle Biopsy Diagnosis and Risk-Controlled Report Generation [19.43990244213587]
  CorePath is a breast-specialized multimodal pathology foundation model fine-tuned from PRISM.<n>CorePath consistently outperformed PRISM across cancer detection, invasion assessment, and histological subtyping.
  arXiv  Detail & Related papers  (2026-08-04T03:43:49Z)
• Blasto-Net: An Explainable Multi-Task Learning for Blastocyst Segmentation, Grading, and Implantation Prediction [1.948638796030649]
  Blasto-Net is a multi-task deep learning model for comprehensive blastocyst analysis.<n>Model performs three tasks simultaneously in a single forward pass: segmentation of the ZP, TE, and ICM compartments, morphological grading, and implantation outcome prediction.
  arXiv  Detail & Related papers  (2026-06-24T06:45:06Z)
• OncoTraj: a public benchmark for longitudinal resistance prediction in EGFR-mutant non-small-cell lung cancer on osimertinib [0.0]
  OncoTraj is a public benchmark of 813 EGFR-mutant NSCLC patients receiving first-line osimertinib.<n>OncoTraj defines three locked tasks: (A) binary classification of progression by a fixed 12-month landmark, (B) regression of time-to-first-progression in days, and (C) six-class classification of the dominant resistance mechanism.
  arXiv  Detail & Related papers  (2026-06-09T17:33:24Z)
• TriALS: Triphasic-Aided Liver Lesion Segmentation Benchmark in Non-Contrast CT [49.93685997940356]
  We describe the TriALS challenge for automated liver lesion segmentation under contrast-limited conditions.<n>The top-performing method achieved a mean venous-phase Dice of 0.754, consistent with human-level performance, yet dropped to 0.57 on NCCT.<n>Algorithm performance was most strongly predicted by training data scale and pre-training strategy.
  arXiv  Detail & Related papers  (2026-05-15T19:23:35Z)
• PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities [86.63247982275396]
  PRIME is a missing-aware multimodal self-supervised pretraining framework.<n>It learns robust and transferable representations from partially observed cohorts.<n>We evaluate PRIME on The Cancer Genome Atlas with label-free pretraining on 32 cancer types.
  arXiv  Detail & Related papers  (2026-04-05T21:14:27Z)
• FUGC: Benchmarking Semi-Supervised Learning Methods for Cervical Segmentation [63.7829089874007]
  This paper introduces the Fetal Ultrasound Grand Challenge (FUGC), the first benchmark for semi-supervised learning in cervical segmentation.<n>FUGC provides a dataset of 890 TVS images, including 500 training images, 90 validation images, and 300 test images.<n> Methods were evaluated using the Dice Similarity Coefficient (DSC), Hausdorff Distance (HD), and runtime (RT), with a weighted combination of 0.4/0.4/0.2.
  arXiv  Detail & Related papers  (2026-01-22T01:34:39Z)
• A decoupled alignment kernel for peptide membrane permeability predictions [35.849562641740754]
  We propose a monomer-aware decoupled global alignment kernel (MD-GAK), which couples chemically meaningful residue-residue similarity with sequence alignment.<n>We also introduce a variant, PMD-GAK, which incorporates a triangular positional prior.<n>Since our focus is on uncertainty estimation, we use Gaussian Processes as the predictive model, as both MD-GAK and PMD-GAK can be directly applied within this framework.
  arXiv  Detail & Related papers  (2025-11-26T16:40:41Z)
• DRBD-Mamba for Robust and Efficient Brain Tumor Segmentation with Analytical Insights [54.87947751720332]
  Accurate brain tumor segmentation is significant for clinical diagnosis and treatment.<n>Mamba-based State Space Models have demonstrated promising performance.<n>We propose a dual-resolution bi-directional Mamba that captures multi-scale long-range dependencies with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-10-16T07:31:21Z)
• Hemorica: A Comprehensive CT Scan Dataset for Automated Brain Hemorrhage Classification, Segmentation, and Detection [0.749500254646884]
  Hemorica is a publicly available collection of 372 head CT examinations acquired between 2012 and 2024.<n>Each scan has been exhaustively annotated for five ICH subtypes-epidural (EPH), subdural (SDH), subarachnoid (SAH), intraparenchymal (IPH)<n>Hemorica offers a unified, fine-grained benchmark that supports multi-task and curriculum learning.
  arXiv  Detail & Related papers  (2025-09-26T23:09:41Z)
• Leveraging Fine-Tuned Large Language Models for Interpretable Pancreatic Cystic Lesion Feature Extraction and Risk Categorization [9.840625513935343]
  Manual extraction of pancreatic cystic lesion (PCL) features from radiology reports is labor-intensive.<n>To develop and evaluate large language models (LLMs) that automatically extract PCL features from MRI/CT reports.
  arXiv  Detail & Related papers  (2025-07-26T15:02:32Z)
• Depth-Sequence Transformer (DST) for Segment-Specific ICA Calcification Mapping on Non-Contrast CT [38.85617601239779]
  Conventional 3D models are forced to process downsampled volumes or isolated patches.<n>We reformulate the 3D challenge as a textbfParallel Probabilistic Landmark localization task along the 1D axial dimension.<n>We propose the textbfDepth-Sequence Transformer (DST), a framework that processes full-resolution CT volumes as sequences of 2D slices.
  arXiv  Detail & Related papers  (2025-07-10T23:12:12Z)
• Segmentation of Bruch's Membrane in retinal OCT with AMD using anatomical priors and uncertainty quantification [4.5206601127476445]
  We propose an end-to-end deep learning method for automated Bruch's membrane (BM) segmentation in AMD patients. An Attention U-Net is trained to output a probability density function of the BM position, while taking into account the natural curvature of the surface. Besides the surface position, the method also estimates an A-scan wise uncertainty measure of the segmentation output.
  arXiv  Detail & Related papers  (2022-10-26T15:49:07Z)
• Hepatic vessel segmentation based on 3Dswin-transformer with inductive biased multi-head self-attention [46.46365941681487]
  We propose a robust end-to-end vessel segmentation network called Indu BIased Multi-Head Attention Vessel Net. We introduce the voxel-wise embedding rather than patch-wise embedding to locate precise liver vessel voxels. On the other hand, we propose inductive biased multi-head self-attention which learns inductive biased relative positional embedding from absolute position embedding.
  arXiv  Detail & Related papers  (2021-11-05T10:17:08Z)
• Nested-block self-attention for robust radiotherapy planning segmentation [3.2541650155921142]
  Deep convolutional networks have been widely studied for head and neck (HN) organs at risk (OAR) segmentation. Their use for routine clinical treatment planning is limited by a lack of robustness to imaging artifacts, low soft tissue contrast on CT, and the presence of abnormal anatomy. We developed a computationally efficient nested block self-attention (NBSA) method that can be combined with any convolutional network.
  arXiv  Detail & Related papers  (2021-02-26T15:28:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.