Bio2Token: All-atom tokenization of any biomolecular structure with Mamba

• URL: http://arxiv.org/abs/2410.19110v1
• Date: Thu, 24 Oct 2024 19:23:09 GMT
• Title: Bio2Token: All-atom tokenization of any biomolecular structure with Mamba
• Authors: Andrew Liu, Axel Elaldi, Nathan Russell, Olivia Viessmann,
• Abstract summary: We develop quantized auto-encoders that learn atom-level tokenizations of complete proteins, RNA and small molecule structures with reconstruction accuracies below and around 1 Angstrom. We demonstrate that the Mamba state space model architecture employed is comparatively efficient, requiring a fraction of the training data, parameters and compute needed to reach competitive accuracies and can scale to systems with almost 100,000 atoms.
• Score: 3.039173168183899
• License:
• Abstract: Efficient encoding and representation of large 3D molecular structures with high fidelity is critical for biomolecular design applications. Despite this, many representation learning approaches restrict themselves to modeling smaller systems or use coarse-grained approximations of the systems, for example modeling proteins at the resolution of amino acid residues rather than at the level of individual atoms. To address this, we develop quantized auto-encoders that learn atom-level tokenizations of complete proteins, RNA and small molecule structures with reconstruction accuracies below and around 1 Angstrom. We demonstrate that the Mamba state space model architecture employed is comparatively efficient, requiring a fraction of the training data, parameters and compute needed to reach competitive accuracies and can scale to systems with almost 100,000 atoms. The learned structure tokens of bio2token may serve as the input for all-atom language models in the future.

Related papers

• GraphXForm: Graph transformer for computer-aided molecular design with application to extraction [73.1842164721868]
  We present GraphXForm, a decoder-only graph transformer architecture, which is pretrained on existing compounds and then fine-tuned. We evaluate it on two solvent design tasks for liquid-liquid extraction, showing that it outperforms four state-of-the-art molecular design techniques.
  arXiv  Detail & Related papers  (2024-11-03T19:45:15Z)
• CryoChains: Heterogeneous Reconstruction of Molecular Assembly of Semi-flexible Chains from Cryo-EM Images [3.0828074702828623]
  We propose CryoChains that encodes large deformations of biomolecules via rigid body transformation of their chains. Our data experiments on the human GABAtextsubscriptB and heat shock protein show that CryoChains gives a biophysically-grounded quantification of the heterogeneous conformations of biomolecules.
  arXiv  Detail & Related papers  (2023-06-12T17:57:12Z)
• Towards Predicting Equilibrium Distributions for Molecular Systems with Deep Learning [60.02391969049972]
  We introduce a novel deep learning framework, called Distributional Graphormer (DiG), in an attempt to predict the equilibrium distribution of molecular systems. DiG employs deep neural networks to transform a simple distribution towards the equilibrium distribution, conditioned on a descriptor of a molecular system.
  arXiv  Detail & Related papers  (2023-06-08T17:12:08Z)
• Bi-level Contrastive Learning for Knowledge-Enhanced Molecule Representations [55.42602325017405]
  We propose a novel method called GODE, which takes into account the two-level structure of individual molecules. By pre-training two graph neural networks (GNNs) on different graph structures, combined with contrastive learning, GODE fuses molecular structures with their corresponding knowledge graph substructures. When fine-tuned across 11 chemical property tasks, our model outperforms existing benchmarks, registering an average ROC-AUC uplift of 13.8% for classification tasks and an average RMSE/MAE enhancement of 35.1% for regression tasks.
  arXiv  Detail & Related papers  (2023-06-02T15:49:45Z)
• Accurate Machine Learned Quantum-Mechanical Force Fields for Biomolecular Simulations [51.68332623405432]
  Molecular dynamics (MD) simulations allow atomistic insights into chemical and biological processes. Recently, machine learned force fields (MLFFs) emerged as an alternative means to execute MD simulations. This work proposes a general approach to constructing accurate MLFFs for large-scale molecular simulations.
  arXiv  Detail & Related papers  (2022-05-17T13:08:28Z)
• Transferring Chemical and Energetic Knowledge Between Molecular Systems with Machine Learning [5.27145343046974]
  We propose a novel methodology for transferring knowledge obtained from simple molecular systems to a more complex one. We focus on the classification of high and low free-energy states. Our results show a remarkable AUC of 0.92 for transfer learning from tri-alanine to the deca-alanine system.
  arXiv  Detail & Related papers  (2022-05-06T16:21:00Z)
• Scalable Fragment-Based 3D Molecular Design with Reinforcement Learning [68.8204255655161]
  We introduce a novel framework for scalable 3D design that uses a hierarchical agent to build molecules. In a variety of experiments, we show that our agent, guided only by energy considerations, can efficiently learn to produce molecules with over 100 atoms.
  arXiv  Detail & Related papers  (2022-02-01T18:54:24Z)
• A silicon qubit platform for in situ single molecule structure determination [0.7187911114620571]
  Imaging individual conformational instances of generic, inhomogeneous, transient or intrinsically disordered protein systems at the single molecule level in situ is one of the notable challenges in structural biology. Here we tackle the problem by designing a single molecule imaging platform technology embracing the advantages silicon-based spin qubits. We demonstrate through detailed simulation, that this platform enables scalable atomic-level structure-determination of individual molecular systems in native environments.
  arXiv  Detail & Related papers  (2021-12-07T10:42:09Z)
• Message Passing Networks for Molecules with Tetrahedral Chirality [8.391459650489123]
  We develop two custom aggregation functions for message passing neural networks to learn properties of molecules with tetrahedral chirality. Results show modest improvements over a baseline sum aggregator, highlighting opportunities for further architecture development.
  arXiv  Detail & Related papers  (2020-11-24T03:03:09Z)
• Self-Supervised Graph Transformer on Large-Scale Molecular Data [73.3448373618865]
  We propose a novel framework, GROVER, for molecular representation learning. GROVER can learn rich structural and semantic information of molecules from enormous unlabelled molecular data. We pre-train GROVER with 100 million parameters on 10 million unlabelled molecules -- the biggest GNN and the largest training dataset in molecular representation learning.
  arXiv  Detail & Related papers  (2020-06-18T08:37:04Z)
• Hierarchical, rotation-equivariant neural networks to select structural models of protein complexes [6.092214762701847]
  We introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes. Our network substantially improves the identification of accurate structural models among a large set of possible models.
  arXiv  Detail & Related papers  (2020-06-05T20:17:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.