Prototype Guided Post-pretraining for Single-Cell Representation Learning

• URL: http://arxiv.org/abs/2605.07938v1
• Date: Fri, 08 May 2026 16:08:10 GMT
• Title: Prototype Guided Post-pretraining for Single-Cell Representation Learning
• Abstract summary: Single-cell representation learning from gene expression data offers a way to uncover the complex regulatory logic underlying cellular function.<n>Several single-cell pretrained models have recently been proposed that treat genes as tokens and cells as sentences.<n>We introduce CellRefine, a post-pretraining method that operates between the pretraining and fine-tuning stages of a single-cell foundation model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Single-cell representation learning (SCRL) from gene expression data offers a way to uncover the complex regulatory logic underlying cellular function. Inspired by large language models in natural language modeling, several single-cell pretrained models have recently been proposed that treat genes as tokens and cells as sentences. However, these models are fundamentally limited by the long-tailed nature of cell-type distributions and struggle to generalize under covariate shifts in gene expression data. While fine-tuning is often used to mitigate these issues, we observe that performance remains bounded. To address this challenge, we introduce CellRefine, a post-pretraining method that operates between the pretraining and fine-tuning stages of a single-cell foundation model. CellRefine uses a multi-faceted objective that incorporates marker-gene sets as structural priors to guide post-pretraining and refine the latent embedding manifold of cells. Across multiple computational biology tasks, empirical results show that CellRefine consistently improves downstream performance, yielding gains up to 15%.

Related papers

• Lingshu-Cell: A generative cellular world model for transcriptome modeling toward virtual cells [48.885716676667776]
  Lingshu-Cell is a discrete diffusion model that learns transcriptomic state distributions and supports conditional simulation under perturbations.<n>It captures complex transcriptome-wide expression dependencies across approximately 18,000 genes without relying on prior gene selection.<n>It achieves leading performance on the Virtual Cell Challenge H1 genetic perturbation benchmark and in predicting cytokine-induced responses in human PBMCs.
  arXiv  Detail & Related papers  (2026-03-26T09:46:27Z)
• Parameter-free representations outperform single-cell foundation models on downstream benchmarks [0.0]
  Single-cell RNA sequencing (scRNA-seq) data exhibit strong and reproducible statistical structure.<n>Large-scale foundation models, such as TranscriptFormer, learn a generative model for gene expression by embedding genes into a latent vector space.<n>We ask whether similar performance can be achieved without utilizing computationally intensive deep learning-based representations.
  arXiv  Detail & Related papers  (2026-02-18T18:42:29Z)
• Scalable Single-Cell Gene Expression Generation with Latent Diffusion Models [11.343106383645441]
  We introduce a scalable latent diffusion model for single-cell gene expression data, which we refer to as scLDM.<n>We show its superior performance in a variety of experiments for both observational and perturbational single-cell data, as well as downstream tasks like cell-level classification.
  arXiv  Detail & Related papers  (2025-11-04T20:44:12Z)
• Cell2Text: Multimodal LLM for Generating Single-Cell Descriptions from RNA-Seq Data [17.440176654185095]
  We introduce Cell2Text, a framework that translates scRNA-seq profiles into structured natural language descriptions.<n>By integrating gene-level embeddings with pretrained large language models, Cell2Text generates coherent summaries that capture cellular identity, tissue origin, disease associations, and pathway activity.
  arXiv  Detail & Related papers  (2025-09-29T14:20:50Z)
• CellVerse: Do Large Language Models Really Understand Cell Biology? [74.34984441715517]
  We introduce CellVerse, a unified language-centric question-answering benchmark that integrates four types of single-cell multi-omics data.<n>We systematically evaluate the performance across 14 open-source and closed-source LLMs ranging from 160M to 671B on CellVerse.
  arXiv  Detail & Related papers  (2025-05-09T06:47:23Z)
• A scalable gene network model of regulatory dynamics in single cells [88.48246132084441]
  We introduce a Functional Learnable model of Cell dynamicS, FLeCS, that incorporates gene network structure into coupled differential equations to model gene regulatory functions.<n>Given (pseudo)time-series single-cell data, FLeCS accurately infers cell dynamics at scale.
  arXiv  Detail & Related papers  (2025-03-25T19:19:21Z)
• scReader: Prompting Large Language Models to Interpret scRNA-seq Data [12.767105992391555]
  We propose an innovative hybrid approach that integrates the general knowledge capabilities of large language models with domain-specific representation models for single-cell omics data interpretation.<n>By inputting single-cell gene-level expression data with prompts, we effectively model cellular representations based on the differential expression levels of genes across various species and cell types.
  arXiv  Detail & Related papers  (2024-12-24T04:28:42Z)
• Multi-Modal and Multi-Attribute Generation of Single Cells with CFGen [76.02070962797794]
  This work introduces CellFlow for Generation (CFGen), a flow-based conditional generative model that preserves the inherent discreteness of single-cell data.<n>CFGen generates whole-genome multi-modal single-cell data reliably, improving the recovery of crucial biological data characteristics.
  arXiv  Detail & Related papers  (2024-07-16T14:05:03Z)
• Mixed Models with Multiple Instance Learning [51.440557223100164]
  We introduce MixMIL, a framework integrating Generalized Linear Mixed Models (GLMM) and Multiple Instance Learning (MIL) Our empirical results reveal that MixMIL outperforms existing MIL models in single-cell datasets.
  arXiv  Detail & Related papers  (2023-11-04T16:42:42Z)
• Multi-modal Self-supervised Pre-training for Regulatory Genome Across Cell Types [75.65676405302105]
  We propose a simple yet effective approach for pre-training genome data in a multi-modal and self-supervised manner, which we call GeneBERT. We pre-train our model on the ATAC-seq dataset with 17 million genome sequences.
  arXiv  Detail & Related papers  (2021-10-11T12:48:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.