One protein is all you need

• URL: http://arxiv.org/abs/2411.02109v2
• Date: Mon, 20 Oct 2025 22:44:43 GMT
• Title: One protein is all you need
• Authors: Anton Bushuiev, Roman Bushuiev, Olga Pimenova, Nikola Zadorozhny, Raman Samusevich, Elisabet Manaskova, Rachel Seongeun Kim, Hannes Stärk, Jiri Sedlar, Martin Steinegger, Tomáš Pluskal, Josef Sivic,
• Abstract summary: Generalization beyond training data remains a central challenge in machine learning for biology.<n>We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets.
• Score: 17.551862138613405
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalization beyond training data remains a central challenge in machine learning for biology. A common way to enhance generalization is self-supervised pre-training on large datasets. However, aiming to perform well on all possible proteins can limit a model's capacity to excel on any specific one, whereas experimentalists typically need accurate predictions for individual proteins they study, often not covered in training data. To address this limitation, we propose a method that enables self-supervised customization of protein language models to one target protein at a time, on the fly, and without assuming any additional data. We show that our Protein Test-Time Training (ProteinTTT) method consistently enhances generalization across different models, their sizes, and datasets. ProteinTTT improves structure prediction for challenging targets, achieves new state-of-the-art results on protein fitness prediction, and enhances function prediction on two tasks. Through two challenging case studies, we also show that customization via ProteinTTT achieves more accurate antibody-antigen loop modeling and enhances 19% of structures in the Big Fantastic Virus Database, delivering improved predictions where general-purpose AlphaFold2 and ESMFold struggle.

Related papers

• Few-shot Protein Fitness Prediction via In-context Learning and Test-time Training [16.300177286668475]
  PRIMO is a transformer-based framework that leverages in-context learning and test-time training to adapt rapidly to new proteins.<n>PRIMO learns to prioritize promising variants through a preference-based loss function.<n>This work underscores the power of combining large-scale pre-training with efficient test-time adaptation to tackle challenging protein design tasks.
  arXiv  Detail & Related papers  (2025-12-02T01:20:40Z)
• Exploring zero-shot structure-based protein fitness prediction [0.5524804393257919]
  We make zero-shot predictions about the fitness consequences of protein sequence changes with pre-trained machine learning models. We assess several modeling choices for structure-based models and their effects on downstream fitness prediction.
  arXiv  Detail & Related papers  (2025-04-23T17:01:09Z)
• Protein Large Language Models: A Comprehensive Survey [71.65899614084853]
  Protein-specific large language models (Protein LLMs) are revolutionizing protein science by enabling more efficient protein structure prediction, function annotation, and design.<n>This work provides the first comprehensive overview of Protein LLMs, covering their architectures, training datasets, evaluation metrics, and diverse applications.
  arXiv  Detail & Related papers  (2025-02-21T19:22:10Z)
• SFM-Protein: Integrative Co-evolutionary Pre-training for Advanced Protein Sequence Representation [97.99658944212675]
  We introduce a novel pre-training strategy for protein foundation models. It emphasizes the interactions among amino acid residues to enhance the extraction of both short-range and long-range co-evolutionary features. Trained on a large-scale protein sequence dataset, our model demonstrates superior generalization ability.
  arXiv  Detail & Related papers  (2024-10-31T15:22:03Z)
• Metalic: Meta-Learning In-Context with Protein Language Models [5.868595531658237]
  Machine learning has emerged as a promising technique for such prediction tasks. Due to data scarcity, we believe meta-learning will play a pivotal role in advancing protein engineering.
  arXiv  Detail & Related papers  (2024-10-10T20:19:35Z)
• Protein-Mamba: Biological Mamba Models for Protein Function Prediction [18.642511763423048]
  Protein-Mamba is a novel two-stage model that leverages both self-supervised learning and fine-tuning to improve protein function prediction. Our experiments demonstrate that Protein-Mamba achieves competitive performance, compared with a couple of state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-09-22T22:51:56Z)
• ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training [82.37346937497136]
  We propose a versatile cross-modal large language model (LLM) for both protein-centric and protein-language tasks. ProtLLM features a unique dynamic protein mounting mechanism, enabling it to handle complex inputs. By developing a specialized protein vocabulary, we equip the model with the capability to predict not just natural language but also proteins from a vast pool of candidates.
  arXiv  Detail & Related papers  (2024-02-28T01:29:55Z)
• xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein [74.64101864289572]
  We propose a unified protein language model, xTrimoPGLM, to address protein understanding and generation tasks simultaneously.<n>xTrimoPGLM significantly outperforms other advanced baselines in 18 protein understanding benchmarks across four categories.<n>It can also generate de novo protein sequences following the principles of natural ones, and can perform programmable generation after supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-01-11T15:03:17Z)
• Efficiently Predicting Protein Stability Changes Upon Single-point Mutation with Large Language Models [51.57843608615827]
  The ability to precisely predict protein thermostability is pivotal for various subfields and applications in biochemistry. We introduce an ESM-assisted efficient approach that integrates protein sequence and structural features to predict the thermostability changes in protein upon single-point mutations.
  arXiv  Detail & Related papers  (2023-12-07T03:25:49Z)
• DeepGATGO: A Hierarchical Pretraining-Based Graph-Attention Model for Automatic Protein Function Prediction [4.608328575930055]
  Automatic protein function prediction (AFP) is classified as a large-scale multi-label classification problem. Currently, popular methods primarily combine protein-related information and Gene Ontology (GO) terms to generate final functional predictions. We propose a sequence-based hierarchical prediction method, DeepGATGO, which processes protein sequences and GO term labels hierarchically.
  arXiv  Detail & Related papers  (2023-07-24T07:01:32Z)
• CCPL: Cross-modal Contrastive Protein Learning [47.095862120116976]
  We introduce a novel unsupervised protein structure representation pretraining method, cross-modal contrastive protein learning (CCPL) CCPL leverages a robust protein language model and uses unsupervised contrastive alignment to enhance structure learning. We evaluated our model across various benchmarks, demonstrating the framework's superiority.
  arXiv  Detail & Related papers  (2023-03-19T08:19:10Z)
• Reprogramming Pretrained Language Models for Protein Sequence Representation Learning [68.75392232599654]
  We propose Representation Learning via Dictionary Learning (R2DL), an end-to-end representation learning framework. R2DL reprograms a pretrained English language model to learn the embeddings of protein sequences. Our model can attain better accuracy and significantly improve the data efficiency by up to $105$ times over the baselines set by pretrained and standard supervised methods.
  arXiv  Detail & Related papers  (2023-01-05T15:55:18Z)
• SESNet: sequence-structure feature-integrated deep learning method for data-efficient protein engineering [6.216757583450049]
  We develop SESNet, a supervised deep-learning model to predict the fitness for protein mutants. We show that SESNet outperforms state-of-the-art models for predicting the sequence-function relationship. Our model can achieve strikingly high accuracy in prediction of the fitness of protein mutants, especially for the higher order variants.
  arXiv  Detail & Related papers  (2022-12-29T01:49:52Z)
• PSP: Million-level Protein Sequence Dataset for Protein Structure Prediction [34.11168458572554]
  We present the first million-level protein structure prediction dataset with high coverage and diversity, named as PSP. This dataset consists of 570k true structure sequences (10TB) and 745k complementary distillation sequences (15TB) We provide in addition the benchmark training procedure for SOTA protein structure prediction model on this dataset.
  arXiv  Detail & Related papers  (2022-06-24T14:08:44Z)
• Structure-aware Protein Self-supervised Learning [50.04673179816619]
  We propose a novel structure-aware protein self-supervised learning method to capture structural information of proteins. In particular, a well-designed graph neural network (GNN) model is pretrained to preserve the protein structural information. We identify the relation between the sequential information in the protein language model and the structural information in the specially designed GNN model via a novel pseudo bi-level optimization scheme.
  arXiv  Detail & Related papers  (2022-04-06T02:18:41Z)
• Protein Representation Learning by Geometric Structure Pretraining [27.723095456631906]
  Existing approaches usually pretrain protein language models on a large number of unlabeled amino acid sequences. We first present a simple yet effective encoder to learn protein geometry features. Experimental results on both function prediction and fold classification tasks show that our proposed pretraining methods outperform or are on par with the state-of-the-art sequence-based methods using much less data.
  arXiv  Detail & Related papers  (2022-03-11T17:52:13Z)
• Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design [56.86533144730384]
  We introduce a method to quantify predictive uncertainty in settings where the training and test data are statistically dependent. As a motivating use case, we demonstrate with several real data sets how our method quantifies uncertainty for the predicted fitness of designed proteins.
  arXiv  Detail & Related papers  (2022-02-08T02:59:12Z)
• EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Models [53.17320541056843]
  We propose a fully-differentiable approach for protein structure optimization, guided by a data-driven generative network. Our EBM-Fold approach can efficiently produce high-quality decoys, compared against traditional Rosetta-based structure optimization routines.
  arXiv  Detail & Related papers  (2021-05-11T03:40:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.