Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data

• URL: http://arxiv.org/abs/2506.11182v1
• Date: Thu, 12 Jun 2025 16:15:14 GMT
• Title: Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data
• Authors: Azim Dehghani Amirabad, Yanfei Zhang, Artem Moskalev, Sowmya Rajesh, Tommaso Mansi, Shuwei Li, Mangal Prakash, Rui Liao,
• Abstract summary: We investigate whether pre-trained biological foundation model originally trained on transcriptomic data can improve gRNA activity estimation.<n>Using embeddings from existing RNA foundation model as input to lightweight regressor, we show substantial gains over traditional baselines.<n>Our results highlight the effectiveness of pre-trained foundation models and accessibility data for gRNA activity prediction.
• Score: 5.002699100842828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting guide RNA (gRNA) activity is critical for effective CRISPR-Cas12 genome editing but remains challenging due to limited data, variation across protospacer adjacent motifs (PAMs-short sequence requirements for Cas binding), and reliance on large-scale training. We investigate whether pre-trained biological foundation model originally trained on transcriptomic data can improve gRNA activity estimation even without domain-specific pre-training. Using embeddings from existing RNA foundation model as input to lightweight regressor, we show substantial gains over traditional baselines. We also integrate chromatin accessibility data to capture regulatory context, improving performance further. Our results highlight the effectiveness of pre-trained foundation models and chromatin accessibility data for gRNA activity prediction.

Related papers

• Regulatory DNA sequence Design with Reinforcement Learning [56.20290878358356]
  We propose a generative approach that leverages reinforcement learning to fine-tune a pre-trained autoregressive model.<n>We evaluate our method on promoter design tasks in two yeast media conditions and enhancer design tasks for three human cell types.
  arXiv  Detail & Related papers  (2025-03-11T02:33:33Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models [0.0]
  understanding and predicting RNA behavior is a challenge due to the complexity of RNA structures and interactions. Current RNA models have yet to match the performance observed in the protein domain. ChaRNABERT is able to reach state-of-the-art performance on several tasks in established benchmarks.
  arXiv  Detail & Related papers  (2024-11-05T21:56:16Z)
• DeepFM-Crispr: Prediction of CRISPR On-Target Effects via Deep Learning [0.24554686192257422]
  DeepFM-Crispr is a novel deep learning model developed to predict the on-target efficiency and evaluate the off-target effects of Cas13d. It harnesses a large language model to generate comprehensive representations rich in evolutionary and structural data, thereby enhancing predictions of RNA secondary structures and overall sgRNA efficacy.
  arXiv  Detail & Related papers  (2024-09-09T17:33:54Z)
• BEACON: Benchmark for Comprehensive RNA Tasks and Language Models [60.02663015002029]
  We introduce the first comprehensive RNA benchmark BEACON (textbfBEnchmtextbfArk for textbfCOmprehensive RtextbfNA Task and Language Models).<n>First, BEACON comprises 13 distinct tasks derived from extensive previous work covering structural analysis, functional studies, and engineering applications.<n>Second, we examine a range of models, including traditional approaches like CNNs, as well as advanced RNA foundation models based on language models, offering valuable insights into the task-specific performances of these models.<n>Third, we investigate the vital RNA language model components
  arXiv  Detail & Related papers  (2024-06-14T19:39:19Z)
• Regressor-free Molecule Generation to Support Drug Response Prediction [83.25894107956735]
  Conditional generation based on the target IC50 score can obtain a more effective sampling space. Regressor-free guidance combines a diffusion model's score estimation with a regression controller model's gradient based on number labels.
  arXiv  Detail & Related papers  (2024-05-23T13:22:17Z)
• Splicing Up Your Predictions with RNA Contrastive Learning [4.35360799431127]
  We extend contrastive learning techniques to genomic data by utilizing similarities between functional sequences generated through alternative splicing gene duplication. We validate their utility on downstream tasks such as RNA half-life and mean ribosome load prediction. Our exploration of the learned latent space reveals that our contrastive objective yields semantically meaningful representations.
  arXiv  Detail & Related papers  (2023-10-12T21:51:25Z)
• How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression? [92.90857135952231]
  Transformers pretrained on diverse tasks exhibit remarkable in-context learning (ICL) capabilities. We study ICL in one of its simplest setups: pretraining a linearly parameterized single-layer linear attention model for linear regression.
  arXiv  Detail & Related papers  (2023-10-12T15:01:43Z)
• Scalable Deep Learning for RNA Secondary Structure Prediction [38.46798525594529]
  We present the RNAformer, a lean deep learning model using axial attention and recycling in the latent space. Our approach achieves state-of-the-art performance on the popular TS0 benchmark dataset. We show experimentally that the RNAformer can learn a biophysical model of the RNA folding process.
  arXiv  Detail & Related papers  (2023-07-14T12:54:56Z)
• Reprogramming Pretrained Language Models for Antibody Sequence Infilling [72.13295049594585]
  Computational design of antibodies involves generating novel and diverse sequences, while maintaining structural consistency. Recent deep learning models have shown impressive results, however the limited number of known antibody sequence/structure pairs frequently leads to degraded performance. In our work we address this challenge by leveraging Model Reprogramming (MR), which repurposes pretrained models on a source language to adapt to the tasks that are in a different language and have scarce data.
  arXiv  Detail & Related papers  (2022-10-05T20:44:55Z)
• Accurate RNA 3D structure prediction using a language model-based deep learning approach [50.193512039121984]
  RhoFold+ is an RNA language model-based deep learning method that accurately predicts 3D structures of single-chain RNAs from sequences.<n>RhoFold+ offers a fully automated end-to-end pipeline for RNA 3D structure prediction.
  arXiv  Detail & Related papers  (2022-07-04T17:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.