Protein-Mamba: Biological Mamba Models for Protein Function Prediction

• URL: http://arxiv.org/abs/2409.14617v1
• Date: Sun, 22 Sep 2024 22:51:56 GMT
• Title: Protein-Mamba: Biological Mamba Models for Protein Function Prediction
• Authors: Bohao Xu, Yingzhou Lu, Yoshitaka Inoue, Namkyeong Lee, Tianfan Fu, Jintai Chen,
• Abstract summary: Protein-Mamba is a novel two-stage model that leverages both self-supervised learning and fine-tuning to improve protein function prediction. Our experiments demonstrate that Protein-Mamba achieves competitive performance, compared with a couple of state-of-the-art methods.
• Score: 18.642511763423048
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Protein function prediction is a pivotal task in drug discovery, significantly impacting the development of effective and safe therapeutics. Traditional machine learning models often struggle with the complexity and variability inherent in predicting protein functions, necessitating more sophisticated approaches. In this work, we introduce Protein-Mamba, a novel two-stage model that leverages both self-supervised learning and fine-tuning to improve protein function prediction. The pre-training stage allows the model to capture general chemical structures and relationships from large, unlabeled datasets, while the fine-tuning stage refines these insights using specific labeled datasets, resulting in superior prediction performance. Our extensive experiments demonstrate that Protein-Mamba achieves competitive performance, compared with a couple of state-of-the-art methods across a range of protein function datasets. This model's ability to effectively utilize both unlabeled and labeled data highlights the potential of self-supervised learning in advancing protein function prediction and offers a promising direction for future research in drug discovery.

Related papers

• Training on test proteins improves fitness, structure, and function prediction [18.176929152066872]
  Self-supervised pre-training on large datasets is a common method to enhance generalization. We introduce a method for self-supervised fine-tuning at test time, allowing models to adapt to the test protein of interest on the fly. We show that our method leads to new state-of-the-art results on the standard benchmark for protein fitness prediction.
  arXiv  Detail & Related papers  (2024-11-04T14:23:59Z)
• SFM-Protein: Integrative Co-evolutionary Pre-training for Advanced Protein Sequence Representation [97.99658944212675]
  We introduce a novel pre-training strategy for protein foundation models. It emphasizes the interactions among amino acid residues to enhance the extraction of both short-range and long-range co-evolutionary features. Trained on a large-scale protein sequence dataset, our model demonstrates superior generalization ability.
  arXiv  Detail & Related papers  (2024-10-31T15:22:03Z)
• Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clustering [24.415612744612773]
  Proteins are essential to life's processes, underpinning evolution and diversity. Advances in sequencing technology have revealed millions of proteins, underscoring the need for sophisticated pre-trained protein models for biological analysis and AI development. Facebook's ESM2, the most advanced protein language model to date, leverages a masked prediction task for unsupervised learning, crafting amino acid representations with notable biochemical accuracy. Yet, it lacks in delivering functional protein insights, signaling an opportunity for enhancing representation quality. This study addresses this gap by incorporating protein family classification into ESM2's training, while a contextual prediction task fine-tunes local
  arXiv  Detail & Related papers  (2024-04-24T11:09:43Z)
• ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction [54.132290875513405]
  The prediction of protein-protein interactions (PPIs) is crucial for understanding biological functions and diseases. Previous machine learning approaches to PPI prediction mainly focus on direct physical interactions. We propose a novel framework ProLLM that employs an LLM tailored for PPI for the first time.
  arXiv  Detail & Related papers  (2024-03-30T05:32:42Z)
• Enhancing Protein Predictive Models via Proteins Data Augmentation: A Benchmark and New Directions [58.819567030843025]
  This paper extends data augmentation techniques previously used for images and texts to proteins and then benchmarks these techniques on a variety of protein-related tasks. We propose two novel semantic-level protein augmentation methods, namely Integrated Gradients Substitution and Back Translation Substitution. Finally, we integrate extended and proposed augmentations into an augmentation pool and propose a simple but effective framework, namely Automated Protein Augmentation (APA)
  arXiv  Detail & Related papers  (2024-03-01T07:58:29Z)
• ProtIR: Iterative Refinement between Retrievers and Predictors for Protein Function Annotation [38.019425619750265]
  We introduce a novel variational pseudo-likelihood framework, ProtIR, designed to improve function predictors by incorporating inter-protein similarity modeling. ProtIR showcases around 10% improvement over vanilla predictor-based methods. It achieves performance on par with protein language model-based methods, yet without the need for massive pre-training.
  arXiv  Detail & Related papers  (2024-02-10T17:31:46Z)
• Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Models [51.57843608615827]
  The ability to precisely predict protein thermostability is pivotal for various subfields and applications in biochemistry. We introduce an ESM-assisted efficient approach that integrates protein sequence and structural features to predict the thermostability changes in protein upon single-point mutations.
  arXiv  Detail & Related papers  (2023-12-07T03:25:49Z)
• Multi-level Protein Representation Learning for Blind Mutational Effect Prediction [5.207307163958806]
  This paper introduces a novel pre-training framework that cascades sequential and geometric analyzers for protein structures. It guides mutational directions toward desired traits by simulating natural selection on wild-type proteins. We assess the proposed approach using a public database and two new databases for a variety of variant effect prediction tasks.
  arXiv  Detail & Related papers  (2023-06-08T03:00:50Z)
• A Latent Diffusion Model for Protein Structure Generation [50.74232632854264]
  We propose a latent diffusion model that can reduce the complexity of protein modeling. We show that our method can effectively generate novel protein backbone structures with high designability and efficiency.
  arXiv  Detail & Related papers  (2023-05-06T19:10:19Z)
• Structure-aware Protein Self-supervised Learning [50.04673179816619]
  We propose a novel structure-aware protein self-supervised learning method to capture structural information of proteins. In particular, a well-designed graph neural network (GNN) model is pretrained to preserve the protein structural information. We identify the relation between the sequential information in the protein language model and the structural information in the specially designed GNN model via a novel pseudo bi-level optimization scheme.
  arXiv  Detail & Related papers  (2022-04-06T02:18:41Z)
• Protein Representation Learning by Geometric Structure Pretraining [27.723095456631906]
  Existing approaches usually pretrain protein language models on a large number of unlabeled amino acid sequences. We first present a simple yet effective encoder to learn protein geometry features. Experimental results on both function prediction and fold classification tasks show that our proposed pretraining methods outperform or are on par with the state-of-the-art sequence-based methods using much less data.
  arXiv  Detail & Related papers  (2022-03-11T17:52:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.