Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction

• URL: http://arxiv.org/abs/2412.13478v1
• Date: Wed, 18 Dec 2024 03:42:20 GMT
• Title: Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction
• Authors: Sepideh Maleki, Jan-Christian Huetter, Kangway V. Chuang, Gabriele Scalia, Tommaso Biancalani,
• Abstract summary: In this study, we leverage single-cell foundation models (FMs) pre-trained on tens of millions of single cells. We introduce a drug-conditional adapter that allows efficient fine-tuning by training less than 1% of the original foundation model.
• Score: 0.6501158610800594
• License:
• Abstract: Predicting transcriptional responses to novel drugs provides a unique opportunity to accelerate biomedical research and advance drug discovery efforts. However, the inherent complexity and high dimensionality of cellular responses, combined with the extremely limited available experimental data, makes the task challenging. In this study, we leverage single-cell foundation models (FMs) pre-trained on tens of millions of single cells, encompassing multiple cell types, states, and disease annotations, to address molecular perturbation prediction. We introduce a drug-conditional adapter that allows efficient fine-tuning by training less than 1% of the original foundation model, thus enabling molecular conditioning while preserving the rich biological representation learned during pre-training. The proposed strategy allows not only the prediction of cellular responses to novel drugs, but also the zero-shot generalization to unseen cell lines. We establish a robust evaluation framework to assess model performance across different generalization tasks, demonstrating state-of-the-art results across all settings, with significant improvements in the few-shot and zero-shot generalization to new cell lines compared to existing baselines.

Related papers

• Single-Cell Omics Arena: A Benchmark Study for Large Language Models on Cell Type Annotation Using Single-Cell Data [13.56585855722118]
  Large language models (LLMs) have demonstrated their ability to efficiently process and synthesize vast corpora of text to automatically extract biological knowledge. Our study explores the potential of LLMs to accurately classify and annotate cell types in single-cell RNA sequencing (scRNA-seq) data. The results demonstrate that LLMs can provide robust interpretations of single-cell data without requiring additional fine-tuning.
  arXiv  Detail & Related papers  (2024-12-03T23:58:35Z)
• Generating Multi-Modal and Multi-Attribute Single-Cell Counts with CFGen [76.02070962797794]
  We present Cell Flow for Generation, a flow-based conditional generative model for multi-modal single-cell counts. Our results suggest improved recovery of crucial biological data characteristics while accounting for novel generative tasks.
  arXiv  Detail & Related papers  (2024-07-16T14:05:03Z)
• Towards generalization of drug response prediction to single cells and patients utilizing importance-aware multi-source domain transfer learning [6.810949980810495]
  scAdaDrug is a novel multi-source domain adaptation model powered by adaptive importance-aware representation learning. Our model achieved state-of-the-art performance in predicting drug response on multiple independent datasets.
  arXiv  Detail & Related papers  (2024-03-08T12:31:03Z)
• Regression-Based Analysis of Multimodal Single-Cell Data Integration Strategies [0.0]
  Multimodal single-cell technologies enable the simultaneous collection of diverse data types from individual cells. This study highlights the exceptional performance of Echo State Networks, boasting a remarkable correlation score of 0.94. These findings hold promise for advancing comprehension of cellular differentiation and function, leveraging the potential of Machine Learning.
  arXiv  Detail & Related papers  (2023-11-21T16:31:27Z)
• Causal machine learning for single-cell genomics [94.28105176231739]
  We discuss the application of machine learning techniques to single-cell genomics and their challenges. We first present the model that underlies most of current causal approaches to single-cell biology. We then identify open problems in the application of causal approaches to single-cell data.
  arXiv  Detail & Related papers  (2023-10-23T13:35:24Z)
• Drug Synergistic Combinations Predictions via Large-Scale Pre-Training and Graph Structure Learning [82.93806087715507]
  Drug combination therapy is a well-established strategy for disease treatment with better effectiveness and less safety degradation. Deep learning models have emerged as an efficient way to discover synergistic combinations. Our framework achieves state-of-the-art results in comparison with other deep learning-based methods.
  arXiv  Detail & Related papers  (2023-01-14T15:07:43Z)
• Retrieval-based Controllable Molecule Generation [63.44583084888342]
  We propose a new retrieval-based framework for controllable molecule generation. We use a small set of molecules to steer the pre-trained generative model towards synthesizing molecules that satisfy the given design criteria. Our approach is agnostic to the choice of generative models and requires no task-specific fine-tuning.
  arXiv  Detail & Related papers  (2022-08-23T17:01:16Z)
• Predicting single-cell perturbation responses for unseen drugs [3.4591290171087747]
  We introduce a new encoder-decoder architecture to study the perturbational effects of unseen drugs. We envision that our proposed method will facilitate more efficient experiment designs through its ability to generate in-silico hypotheses.
  arXiv  Detail & Related papers  (2022-04-28T14:53:54Z)
• Multi-modal Self-supervised Pre-training for Regulatory Genome Across Cell Types [75.65676405302105]
  We propose a simple yet effective approach for pre-training genome data in a multi-modal and self-supervised manner, which we call GeneBERT. We pre-train our model on the ATAC-seq dataset with 17 million genome sequences.
  arXiv  Detail & Related papers  (2021-10-11T12:48:44Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.