Mechanisms of AI Protein Folding in ESMFold

• URL: http://arxiv.org/abs/2602.06020v2
• Date: Sun, 08 Feb 2026 20:38:53 GMT
• Title: Mechanisms of AI Protein Folding in ESMFold
• Authors: Kevin Lu, Jannik Brinkmann, Stefan Huber, Aaron Mueller, Yonatan Belinkov, David Bau, Chris Wendler,
• Abstract summary: We trace how ESMFold folds a beta hairpin, a prevalent structural motif.<n>We demonstrate that the mechanisms underlying structural decisions of ESMFold can be localized, traced through interpretable representations, and manipulated with strong causal effects.
• Score: 60.598995350547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: How do protein structure prediction models fold proteins? We investigate this question by tracing how ESMFold folds a beta hairpin, a prevalent structural motif. Through counterfactual interventions on model latents, we identify two computational stages in the folding trunk. In the first stage, early blocks initialize pairwise biochemical signals: residue identities and associated biochemical features such as charge flow from sequence representations into pairwise representations. In the second stage, late blocks develop pairwise spatial features: distance and contact information accumulate in the pairwise representation. We demonstrate that the mechanisms underlying structural decisions of ESMFold can be localized, traced through interpretable representations, and manipulated with strong causal effects.

Related papers

• Dynamics-inspired Structure Hallucination for Protein-protein Interaction Modeling [60.57197355431804]
  Protein-protein interaction (PPI) represents a central challenge within the biology field.<n>Deep learning has shown promise in forecasting the effects of such mutations, but is hindered by two primary constraints.<n>We present a novel framework named Refine-PPI with two key enhancements.
  arXiv  Detail & Related papers  (2026-01-08T19:29:04Z)
• KEPLA: A Knowledge-Enhanced Deep Learning Framework for Accurate Protein-Ligand Binding Affinity Prediction [60.23701115249195]
  KEPLA is a novel deep learning framework that integrates prior knowledge from Gene Ontology and ligand properties to enhance prediction performance.<n> Experiments on two benchmark datasets demonstrate that KEPLA consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-06-16T08:02:42Z)
• Representing local protein environments with atomistic foundation models [6.120694232253299]
  We propose a novel representation for a local protein environment derived from the intermediate features of atomistic foundation models (AFMs)<n>We show that the AFM-derived representation space exhibits meaningful structure, enabling the construction of data-driven priors.<n>In the context of biomolecular NMR spectroscopy, we demonstrate that the proposed representations enable a first-of-its-kind physics-informed chemical shift predictor.
  arXiv  Detail & Related papers  (2025-05-29T11:25:47Z)
• Bidirectional Hierarchical Protein Multi-Modal Representation Learning [4.682021474006426]
  Protein language models (pLMs) pretrained on large scale protein sequences have demonstrated significant success in sequence-based tasks.<n> graph neural networks (GNNs) designed to leverage 3D structural information have shown promising generalization in protein-related prediction tasks.<n>We propose a bidirectional and hierarchical (Bi-Hierarchical) fusion approach to capture richer and more comprehensive protein representations.
  arXiv  Detail & Related papers  (2025-04-07T06:47:49Z)
• The Signed Two-Space Proximity Model for Learning Representations in Protein-Protein Interaction Networks [16.396309363020908]
  Accurately predicting complex protein-protein interactions (PPIs) is crucial for decoding biological processes.<n>We present the Signed Two-Space Proximity Model (S2-SPM) for signed PPI networks, which explicitly incorporates both types of interactions.<n>Our approach also enables the identification of archetypes representing extreme protein profiles.
  arXiv  Detail & Related papers  (2025-03-05T21:08:58Z)
• Diffusion Model with Representation Alignment for Protein Inverse Folding [53.139837825588614]
  Protein inverse folding is a fundamental problem in bioinformatics, aiming to recover the amino acid sequences from a given protein backbone structure.<n>We propose a novel method that leverages diffusion models with representation alignment (DMRA)<n>In experiments, we conduct extensive evaluations on the CATH4.2 dataset to demonstrate that DMRA outperforms leading methods.
  arXiv  Detail & Related papers  (2024-12-12T15:47:59Z)
• ProtSCAPE: Mapping the landscape of protein conformations in molecular dynamics [3.6181745062052495]
  We introduce a novel deep learning architecture, Protein Transformer with Scattering, Attention, and Positional Embedding (ProtSCAPE) ProtSCAPE utilizes the multi-scale nature of the geometric scattering transform to extract features from protein structures conceptualized as graphs. It integrates these features with dual attention structures that focus on residues and amino acid signals, generating latent representations of protein trajectories.
  arXiv  Detail & Related papers  (2024-10-27T02:59:48Z)
• AlphaFolding: 4D Diffusion for Dynamic Protein Structure Prediction with Reference and Motion Guidance [18.90451943620277]
  This study introduces an innovative 4D diffusion model incorporating molecular dynamics (MD) simulation data to learn dynamic protein structures.<n>Our model exhibits high accuracy in predicting dynamic 3D structures of proteins containing up to 256 amino acids over 32 time steps.
  arXiv  Detail & Related papers  (2024-08-22T14:12:50Z)
• State-specific protein-ligand complex structure prediction with a multi-scale deep generative model [68.28309982199902]
  We present NeuralPLexer, a computational approach that can directly predict protein-ligand complex structures. Our study suggests that a data-driven approach can capture the structural cooperativity between proteins and small molecules, showing promise in accelerating the design of enzymes, drug molecules, and beyond.
  arXiv  Detail & Related papers  (2022-09-30T01:46:38Z)
• BERTology Meets Biology: Interpreting Attention in Protein Language Models [124.8966298974842]
  We demonstrate methods for analyzing protein Transformer models through the lens of attention. We show that attention captures the folding structure of proteins, connecting amino acids that are far apart in the underlying sequence, but spatially close in the three-dimensional structure. We also present a three-dimensional visualization of the interaction between attention and protein structure.
  arXiv  Detail & Related papers  (2020-06-26T21:50:17Z)
• Explainable Deep Relational Networks for Predicting Compound-Protein Affinities and Contacts [80.69440684790925]
  DeepRelations is a physics-inspired deep relational network with intrinsically explainable architecture. It shows superior interpretability to the state-of-the-art. It boosts the AUPRC of contact prediction 9.5, 16.9, 19.3 and 5.7-fold for the test, compound-unique, protein-unique, and both-unique sets.
  arXiv  Detail & Related papers  (2019-12-29T00:14:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.