GRNFormer: A Biologically-Guided Framework for Integrating Gene Regulatory Networks into RNA Foundation Models

• URL: http://arxiv.org/abs/2503.01682v1
• Date: Mon, 03 Mar 2025 15:56:39 GMT
• Title: GRNFormer: A Biologically-Guided Framework for Integrating Gene Regulatory Networks into RNA Foundation Models
• Authors: Mufan Qiu, Xinyu Hu, Fengwei Zhan, Sukwon Yun, Jie Peng, Ruichen Zhang, Bhavya Kailkhura, Jiekun Yang, Tianlong Chen,
• Abstract summary: We propose a new framework that integrates multi-scale Gene Regulatory Networks (GRNs) inferred from multi-omics data into RNA foundation model training.<n>GRNFormer achieves consistent improvements over state-of-the-art (SoTA) baselines.
• Score: 39.58414436685698
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models for single-cell RNA sequencing (scRNA-seq) have shown promising capabilities in capturing gene expression patterns. However, current approaches face critical limitations: they ignore biological prior knowledge encoded in gene regulatory relationships and fail to leverage multi-omics signals that could provide complementary regulatory insights. In this paper, we propose GRNFormer, a new framework that systematically integrates multi-scale Gene Regulatory Networks (GRNs) inferred from multi-omics data into RNA foundation model training. Our framework introduces two key innovations. First, we introduce a pipeline for constructing hierarchical GRNs that capture regulatory relationships at both cell-type-specific and cell-specific resolutions. Second, we design a structure-aware integration framework that addresses the information asymmetry in GRNs through two technical advances: (1) A graph topological adapter using multi-head cross-attention to weight regulatory relationships dynamically, and (2) a novel edge perturbation strategy that perturb GRNs with biologically-informed co-expression links to augment graph neural network training. Comprehensive experiments have been conducted on three representative downstream tasks across multiple model architectures to demonstrate the effectiveness of GRNFormer. It achieves consistent improvements over state-of-the-art (SoTA) baselines: $3.6\%$ increase in drug response prediction correlation, $9.6\%$ improvement in single-cell drug classification AUC, and $1.1\%$ average gain in gene perturbation prediction accuracy.

Related papers

• A Novel Graph Transformer Framework for Gene Regulatory Network Inference [0.27624021966289597]
  Inference of gene regulatory networks (GRNs) may not always reflect true biological interactions. Most GRN inference methods face several challenges in the network reconstruction phase. We employ autoencoder embeddings to capture gene expression patterns directly from raw data. We embed the prior knowledge from GRN structures transforming them into a text-like representation.
  arXiv  Detail & Related papers  (2025-04-23T06:24:26Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters. Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks. It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Analysis of Gene Regulatory Networks from Gene Expression Using Graph Neural Networks [0.4369058206183195]
  This study explores the use of Graph Neural Networks (GNNs), a powerful approach for modeling graph-structured data like Gene Regulatory Networks (GRNs) The model's adeptness in accurately predicting regulatory interactions and pinpointing key regulators is attributed to advanced attention mechanisms. The integration of GNNs in GRN research is set to pioneer developments in personalized medicine, drug discovery, and our grasp of biological systems.
  arXiv  Detail & Related papers  (2024-09-20T17:16:14Z)
• Gene Regulatory Network Inference from Pre-trained Single-Cell Transcriptomics Transformer with Joint Graph Learning [10.44434676119443]
  Inferring gene regulatory networks (GRNs) from single-cell RNA sequencing (scRNA-seq) data is a complex challenge. In this study, we tackle this challenge by leveraging the single-cell BERT-based pre-trained transformer model (scBERT) We introduce a novel joint graph learning approach that combines the rich contextual representations learned by single-cell language models with the structured knowledge encoded in GRNs.
  arXiv  Detail & Related papers  (2024-07-25T16:42:08Z)
• Inference of dynamical gene regulatory networks from single-cell data with physics informed neural networks [0.0]
  We show how physics-informed neural networks (PINNs) can be used to infer the parameters of predictive, dynamical GRNs. Specifically we study GRNs that exhibit bifurcation behavior and can therefore model cell differentiation.
  arXiv  Detail & Related papers  (2024-01-14T21:43:10Z)
• scBiGNN: Bilevel Graph Representation Learning for Cell Type Classification from Single-cell RNA Sequencing Data [62.87454293046843]
  Graph neural networks (GNNs) have been widely used for automatic cell type classification. scBiGNN comprises two GNN modules to identify cell types. scBiGNN outperforms a variety of existing methods for cell type classification from scRNA-seq data.
  arXiv  Detail & Related papers  (2023-12-16T03:54:26Z)
• Causal Inference in Gene Regulatory Networks with GFlowNet: Towards Scalability in Large Systems [87.45270862120866]
  We introduce Swift-DynGFN as a novel framework that enhances causal structure learning in GRNs. Specifically, Swift-DynGFN exploits gene-wise independence to boost parallelization and to lower computational cost.
  arXiv  Detail & Related papers  (2023-10-05T14:59:19Z)
• Stability Analysis of Non-Linear Classifiers using Gene Regulatory Neural Network for Biological AI [2.0755366440393743]
  We develop a mathematical model of gene-perceptron using a dual-layered transcription-translation chemical reaction model. We perform stability analysis for each gene-perceptron within the fully-connected GRNN sub network to determine temporal as well as stable concentration outputs.
  arXiv  Detail & Related papers  (2023-09-14T21:37:38Z)
• DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets [81.75973217676986]
  Gene regulatory networks (GRN) describe interactions between genes and their products that control gene expression and cellular function. Existing methods either focus on challenge (1), identifying cyclic structure from dynamics, or on challenge (2) learning complex Bayesian posteriors over DAGs, but not both. In this paper we leverage the fact that it is possible to estimate the "velocity" of gene expression with RNA velocity techniques to develop an approach that addresses both challenges.
  arXiv  Detail & Related papers  (2023-02-08T16:36:40Z)
• Granger causal inference on DAGs identifies genomic loci regulating transcription [77.58911272503771]
  GrID-Net is a framework based on graph neural networks with lagged message passing for Granger causal inference on DAG-structured systems. Our application is the analysis of single-cell multimodal data to identify genomic loci that mediate the regulation of specific genes.
  arXiv  Detail & Related papers  (2022-10-18T21:15:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.