Structure Language Models for Protein Conformation Generation

• URL: http://arxiv.org/abs/2410.18403v1
• Date: Thu, 24 Oct 2024 03:38:51 GMT
• Title: Structure Language Models for Protein Conformation Generation
• Authors: Jiarui Lu, Xiaoyin Chen, Stephen Zhewen Lu, Chence Shi, Hongyu Guo, Yoshua Bengio, Jian Tang,
• Abstract summary: Traditional physics-based simulation methods often struggle with sampling equilibrium conformations. Deep generative models have shown promise in generating protein conformations as a more efficient alternative. We introduce Structure Language Modeling as a novel framework for efficient protein conformation generation.
• Score: 66.42864253026053
• License:
• Abstract: Proteins adopt multiple structural conformations to perform their diverse biological functions, and understanding these conformations is crucial for advancing drug discovery. Traditional physics-based simulation methods often struggle with sampling equilibrium conformations and are computationally expensive. Recently, deep generative models have shown promise in generating protein conformations as a more efficient alternative. However, these methods predominantly rely on the diffusion process within a 3D geometric space, which typically centers around the vicinity of metastable states and is often inefficient in terms of runtime. In this paper, we introduce Structure Language Modeling (SLM) as a novel framework for efficient protein conformation generation. Specifically, the protein structures are first encoded into a compact latent space using a discrete variational auto-encoder, followed by conditional language modeling that effectively captures sequence-specific conformation distributions. This enables a more efficient and interpretable exploration of diverse ensemble modes compared to existing methods. Based on this general framework, we instantiate SLM with various popular LM architectures as well as proposing the ESMDiff, a novel BERT-like structure language model fine-tuned from ESM3 with masked diffusion. We verify our approach in various scenarios, including the equilibrium dynamics of BPTI, conformational change pairs, and intrinsically disordered proteins. SLM provides a highly efficient solution, offering a 20-100x speedup than existing methods in generating diverse conformations, shedding light on promising avenues for future research.

Related papers

• DPLM-2: A Multimodal Diffusion Protein Language Model [75.98083311705182]
  We introduce DPLM-2, a multimodal protein foundation model that extends discrete diffusion protein language model (DPLM) to accommodate both sequences and structures. DPLM-2 learns the joint distribution of sequence and structure, as well as their marginals and conditionals. Empirical evaluation shows that DPLM-2 can simultaneously generate highly compatible amino acid sequences and their corresponding 3D structures.
  arXiv  Detail & Related papers  (2024-10-17T17:20:24Z)
• Protein Conformation Generation via Force-Guided SE(3) Diffusion Models [48.48934625235448]
  Deep generative modeling techniques have been employed to generate novel protein conformations. We propose a force-guided SE(3) diffusion model, ConfDiff, for protein conformation generation.
  arXiv  Detail & Related papers  (2024-03-21T02:44:08Z)
• Diffusion on language model embeddings for protein sequence generation [0.5442686600296733]
  We introduce DiMA, a model that leverages continuous diffusion to generate amino acid sequences. We quantitatively illustrate the impact of the design choices that lead to its superior performance. Our approach consistently produces novel, diverse protein sequences that accurately reflect the inherent structural and functional diversity of the protein space.
  arXiv  Detail & Related papers  (2024-03-06T14:15:20Z)
• Diffusion Language Models Are Versatile Protein Learners [75.98083311705182]
  This paper introduces diffusion protein language model (DPLM), a versatile protein language model that demonstrates strong generative and predictive capabilities for protein sequences. We first pre-train scalable DPLMs from evolutionary-scale protein sequences within a generative self-supervised discrete diffusion probabilistic framework. After pre-training, DPLM exhibits the ability to generate structurally plausible, novel, and diverse protein sequences for unconditional generation.
  arXiv  Detail & Related papers  (2024-02-28T18:57:56Z)
• AlphaFold Meets Flow Matching for Generating Protein Ensembles [11.1639408863378]
  We develop a flow-based generative modeling approach for learning and sampling the conformational landscapes of proteins. Our method provides a superior combination of precision and diversity compared to AlphaFold with MSA subsampling. Our method can diversify a static PDB structure with faster wall-clock convergence to certain equilibrium properties than replicate MD trajectories.
  arXiv  Detail & Related papers  (2024-02-07T13:44:47Z)
• Ophiuchus: Scalable Modeling of Protein Structures through Hierarchical Coarse-graining SO(3)-Equivariant Autoencoders [1.8835495377767553]
  Three-dimensional native states of natural proteins display recurring and hierarchical patterns. Traditional graph-based modeling of protein structures is often limited to operate within a single fine-grained resolution. We introduce Ophiuchus, an SO(3)-equivariant coarse-graining model that efficiently operates on all-atom protein structures.
  arXiv  Detail & Related papers  (2023-10-04T01:01:11Z)
• Learning Geometrically Disentangled Representations of Protein Folding Simulations [72.03095377508856]
  This work focuses on learning a generative neural network on a structural ensemble of a drug-target protein. Model tasks involve characterizing the distinct structural fluctuations of the protein bound to various drug molecules. Results show that our geometric learning-based method enjoys both accuracy and efficiency for generating complex structural variations.
  arXiv  Detail & Related papers  (2022-05-20T19:38:00Z)
• EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Models [53.17320541056843]
  We propose a fully-differentiable approach for protein structure optimization, guided by a data-driven generative network. Our EBM-Fold approach can efficiently produce high-quality decoys, compared against traditional Rosetta-based structure optimization routines.
  arXiv  Detail & Related papers  (2021-05-11T03:40:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.