Protein Fold Classification at Scale: Benchmarking and Pretraining

• URL: http://arxiv.org/abs/2605.18552v1
• Date: Mon, 18 May 2026 15:32:34 GMT
• Title: Protein Fold Classification at Scale: Benchmarking and Pretraining
• Abstract summary: Masked Invariant Autoencoders (MiAE) is a self-supervised framework for protein structure representation learning.<n>MiAE scales well and outperforms supervised counterparts and state-of-the-art baselines on TEDBench.
• Score: 26.547016915751964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classifying protein topology is essential for deciphering biological function, but progress is held back by the lack of large-scale benchmarks that avoid duplicates and by models that do not scale well. We introduce TEDBench, a large-scale, non-redundant benchmark for protein fold classification constructed from the Encyclopedia of Domains (TED) and Foldseek-clustered AlphaFold structures. We show that on TEDBench, current protein representation learning methods either require very large models or fail to deliver strong performance. To address this challenge, we propose Masked Invariant Autoencoders (MiAE), a self-supervised framework for protein structure representation learning. MiAE uses an extremely high masking ratio of up to 90% with an $\mathrm{SE(3)}$-invariant encoder and a lightweight decoder that reconstructs backbone coordinates from the latent representation and mask tokens. MiAE scales well and outperforms supervised counterparts and state-of-the-art baselines on TEDBench, establishing a strong recipe for protein fold classification. To test transfer beyond AlphaFold structures, we further benchmark on a curated dataset from experimental structures of CATH v4.4. TEDBench is available at https://github.com/BorgwardtLab/TEDBench.

Related papers

• Hyper-Fold: Exploring the Expressive Limit of Sequence-Geometry Learning for Proteins via Hypergraph Modeling [55.36837305276515]
  We show that the complete bilinear operator over content-geometry outer products--the sufficient statistic of all second-order interactions--is the expressive ceiling.<n>We then introduce Hyper-Fold, a rank-K separable convolutional backbone approaching this ceiling at message-passing cost.
  arXiv  Detail & Related papers  (2026-08-29T11:43:25Z)
• S$^2$Drug: Bridging Protein Sequence and 3D Structure in Contrastive Representation Learning for Virtual Screening [72.89086338778098]
  We propose a two-stage framework for protein-ligand contrastive representation learning.<n>In the first stage, we perform protein sequence pretraining on ChemBL using an ESM2-based backbone.<n>In the second stage, we fine-tune on PDBBind by fusing sequence and structure information through a residue-level gating module.<n>This auxiliary task guides the model to accurately localize binding residues within the protein sequence and capture their 3D spatial arrangement.
  arXiv  Detail & Related papers  (2025-11-10T11:57:47Z)
• ProteinAE: Protein Diffusion Autoencoders for Structure Encoding [64.77182442408254]
  We introduce ProteinAE, a novel and streamlined protein diffusion autoencoder.<n>ProteinAE directly maps protein backbone coordinates from E(3) into a continuous, compact latent space.<n>We demonstrate that ProteinAE achieves state-of-the-art reconstruction quality, outperforming existing autoencoders.
  arXiv  Detail & Related papers  (2025-10-12T14:30:32Z)
• Protein-SE(3): Benchmarking SE(3)-based Generative Models for Protein Structure Design [35.87227562237066]
  SE(3)-based generative models have shown great promise in protein geometry modeling and effective structure design.<n>Protein-SE(3), a new benchmark based on a unified training framework, comprises protein scaffolding tasks, integrated generative models, high-level mathematical abstraction, and diverse evaluation metrics.
  arXiv  Detail & Related papers  (2025-07-27T11:53:05Z)
• Aligning Proteins and Language: A Foundation Model for Protein Retrieval [30.32156711268032]
  This paper aims to retrieve proteins with similar structures and semantics from large-scale protein dataset.<n>Motivated by the recent progress of vision-caption models (VLMs), we propose a CLIP-style framework for aligning 3D protein structures with functional annotations.
  arXiv  Detail & Related papers  (2025-05-27T08:13:08Z)
• Protein Structure Tokenization: Benchmarking and New Recipe [16.842453216446987]
  StructTokenBench is a framework that comprehensively evaluates the quality and efficiency of structure tokenizers.<n>AminoAseed is a strategy that enhances codebook updates and optimally balances codebook size and dimension for improved tokenizer utilization and quality.<n>Our method achieves an average of 6.31% performance improvement across 24 supervised tasks, with sensitivity and utilization rates increased by 12.83% and 124.03%, respectively.
  arXiv  Detail & Related papers  (2025-02-28T15:14:33Z)
• Evaluating representation learning on the protein structure universe [19.856785982914243]
  ProteinWorkshop is a benchmark suite for representation learning on protein structures with Graph Neural Networks. We consider large-scale pre-training and downstream tasks on both experimental and predicted structures. We find that: (1) large-scale pretraining on AlphaFold structures and auxiliary tasks consistently improve the performance of both rotation-invariant and equivariant GNNs.
  arXiv  Detail & Related papers  (2024-06-19T21:48:34Z)
• PDB-Struct: A Comprehensive Benchmark for Structure-based Protein Design [19.324059406159325]
  We introduce two novel metrics: refoldability-based metric and stability-based metric. ByProt, ProteinMPNN, and ESM-IF perform exceptionally well on our benchmark, while ESM-Design and AF-Design fall short. Our proposed benchmark paves the way for a fair and comprehensive evaluation of protein design methods.
  arXiv  Detail & Related papers  (2023-11-30T02:37:55Z)
• High-level Feature Guided Decoding for Semantic Segmentation [54.424062794490254]
  We propose to use powerful pre-trained high-level features as guidance (HFG) for the upsampler to produce robust results. Specifically, the high-level features from the backbone are used to train the class tokens, which are then reused by the upsampler for classification. To push the upper limit of HFG, we introduce a context augmentation encoder (CAE) that can efficiently and effectively operate on the low-resolution high-level feature.
  arXiv  Detail & Related papers  (2023-03-15T14:23:07Z)
• A Systematic Study of Joint Representation Learning on Protein Sequences and Structures [38.94729758958265]
  Learning effective protein representations is critical in a variety of tasks in biology such as predicting protein functions. Recent sequence representation learning methods based on Protein Language Models (PLMs) excel in sequence-based tasks, but their direct adaptation to tasks involving protein structures remains a challenge. Our study undertakes a comprehensive exploration of joint protein representation learning by integrating a state-of-the-art PLM with distinct structure encoders.
  arXiv  Detail & Related papers  (2023-03-11T01:24:10Z)
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning [120.53458753007851]
  Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. We deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse. We propose a neural collapse inspired framework for FSCIL. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances.
  arXiv  Detail & Related papers  (2023-02-06T18:39:40Z)
• HelixFold-Single: MSA-free Protein Structure Prediction by Using Protein Language Model as an Alternative [61.984700682903096]
  HelixFold-Single is proposed to combine a large-scale protein language model with the superior geometric learning capability of AlphaFold2. Our proposed method pre-trains a large-scale protein language model with thousands of millions of primary sequences. We obtain an end-to-end differentiable model to predict the 3D coordinates of atoms from only the primary sequence.
  arXiv  Detail & Related papers  (2022-07-28T07:30:33Z)
• Do We Really Need a Learnable Classifier at the End of Deep Neural Network? [118.18554882199676]
  We study the potential of learning a neural network for classification with the classifier randomly as an ETF and fixed during training. Our experimental results show that our method is able to achieve similar performances on image classification for balanced datasets.
  arXiv  Detail & Related papers  (2022-03-17T04:34:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.