A Fusion-Driven Approach of Attention-Based CNN-BiLSTM for Protein Family Classification -- ProFamNet

• URL: http://arxiv.org/abs/2410.17293v1
• Date: Mon, 21 Oct 2024 20:33:18 GMT
• Title: A Fusion-Driven Approach of Attention-Based CNN-BiLSTM for Protein Family Classification -- ProFamNet
• Authors: Bahar Ali, Anwar Shah, Malik Niaz, Musadaq Mansoord, Sami Ullah, Muhammad Adnan,
• Abstract summary: This study presents a model for classifying protein families using the fusion of 1D-CNN, BiLSTM, and an attention mechanism. The proposed model (ProFamNet) achieved superior model efficiency with 450,953 parameters and a compact size of 1.72 MB.
• Score: 0.8429750290021879
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advanced automated AI techniques allow us to classify protein sequences and discern their biological families and functions. Conventional approaches for classifying these protein families often focus on extracting N-Gram features from the sequences while overlooking crucial motif information and the interplay between motifs and neighboring amino acids. Recently, convolutional neural networks have been applied to amino acid and motif data, even with a limited dataset of well-characterized proteins, resulting in improved performance. This study presents a model for classifying protein families using the fusion of 1D-CNN, BiLSTM, and an attention mechanism, which combines spatial feature extraction, long-term dependencies, and context-aware representations. The proposed model (ProFamNet) achieved superior model efficiency with 450,953 parameters and a compact size of 1.72 MB, outperforming the state-of-the-art model with 4,578,911 parameters and a size of 17.47 MB. Further, we achieved a higher F1 score (98.30% vs. 97.67%) with more instances (271,160 vs. 55,077) in fewer training epochs (25 vs. 30).

Related papers

• SeqProFT: Applying LoRA Finetuning for Sequence-only Protein Property Predictions [8.112057136324431]
  This study employs the LoRA method to perform end-to-end fine-tuning of the ESM-2 model. A multi-head attention mechanism is integrated into the downstream network to combine sequence features with contact map information.
  arXiv  Detail & Related papers  (2024-11-18T12:40:39Z)
• Training Compute-Optimal Protein Language Models [48.79416103951816]
  Most protein language models are trained with extensive compute resources until performance gains plateau. Our investigation is grounded in a massive dataset consisting of 939 million protein sequences. We trained over 300 models ranging from 3.5 million to 10.7 billion parameters on 5 to 200 billion unique tokens.
  arXiv  Detail & Related papers  (2024-11-04T14:58:37Z)
• SFM-Protein: Integrative Co-evolutionary Pre-training for Advanced Protein Sequence Representation [97.99658944212675]
  We introduce a novel pre-training strategy for protein foundation models. It emphasizes the interactions among amino acid residues to enhance the extraction of both short-range and long-range co-evolutionary features. Trained on a large-scale protein sequence dataset, our model demonstrates superior generalization ability.
  arXiv  Detail & Related papers  (2024-10-31T15:22:03Z)
• Protein sequence classification using natural language processing techniques [3.0846824529023396]
  This study employs natural language processing (NLP) techniques on a dataset comprising 75 target protein classes. We explored various machine learning and deep learning models, including K-Nearest Neighbors (KNN), Multinomial Na"ive Bayes, Logistic Regression, Multi-Layer Perceptron (MLP), Decision Tree, Random Forest, XGBoost, Voting and Stacking classifiers, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and transformer models.
  arXiv  Detail & Related papers  (2024-09-06T13:16:16Z)
• Design Proteins Using Large Language Models: Enhancements and Comparative Analyses [12.140433802768733]
  We adopt a suite of pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and gemma-7B4, to produce valid protein sequences. We retrain these models to process protein-related data, ensuring the generation of biologically feasible protein structures. Our findings demonstrate that even with limited data, the adapted models exhibit efficiency comparable to established protein-focused models.
  arXiv  Detail & Related papers  (2024-08-12T08:17:27Z)
• Clustering for Protein Representation Learning [72.72957540484664]
  We propose a neural clustering framework that can automatically discover the critical components of a protein. Our framework treats a protein as a graph, where each node represents an amino acid and each edge represents a spatial or sequential connection between amino acids. We evaluate on four protein-related tasks: protein fold classification, enzyme reaction classification, gene term prediction, and enzyme commission number prediction.
  arXiv  Detail & Related papers  (2024-03-30T05:51:09Z)
• NaNa and MiGu: Semantic Data Augmentation Techniques to Enhance Protein Classification in Graph Neural Networks [60.48306899271866]
  We propose novel semantic data augmentation methods to incorporate backbone chemical and side-chain biophysical information into protein classification tasks. Specifically, we leverage molecular biophysical, secondary structure, chemical bonds, andionic features of proteins to facilitate classification tasks.
  arXiv  Detail & Related papers  (2024-03-21T13:27:57Z)
• UNet-2022: Exploring Dynamics in Non-isomorphic Architecture [52.04899592688968]
  We propose a parallel non-isomorphic block that takes the advantages of self-attention and convolution with simple parallelization. We name the resulting U-shape segmentation model as UNet-2022. In experiments, UNet-2022 obviously outperforms its counterparts in a range segmentation tasks.
  arXiv  Detail & Related papers  (2022-10-27T16:00:04Z)
• EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm [111.17100512647619]
  This paper explains the rationality of Vision Transformer by analogy with the proven practical evolutionary algorithm (EA) We propose a novel pyramid EATFormer backbone that only contains the proposed EA-based transformer (EAT) block. Massive quantitative and quantitative experiments on image classification, downstream tasks, and explanatory experiments demonstrate the effectiveness and superiority of our approach.
  arXiv  Detail & Related papers  (2022-06-19T04:49:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.