Modern Hopfield Networks for Few- and Zero-Shot Reaction Prediction

• URL: http://arxiv.org/abs/2104.03279v1
• Date: Wed, 7 Apr 2021 17:35:00 GMT
• Title: Modern Hopfield Networks for Few- and Zero-Shot Reaction Prediction
• Authors: Philipp Seidl, Philipp Renz, Natalia Dyubankova, Paulo Neves, Jonas Verhoeven, J\"org K. Wegner, Sepp Hochreiter, G\"unter Klambauer
• Abstract summary: Computer-assisted synthesis planning (CASP) to realize physical molecules is still in its infancy and lacks a performance level that would enable large-scale molecule discovery. We propose a novel reaction prediction approach that uses a deep learning architecture with modern Hopfield networks (MHNs) that is optimized by contrastive learning. We show that our MHN contrastive learning approach enables few- and zero-shot learning for reaction prediction which, in contrast to previous methods, can deal with rare, single, or even no training example(s) for a reaction.
• Score: 3.885603826656419
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An essential step in the discovery of new drugs and materials is the synthesis of a molecule that exists so far only as an idea to test its biological and physical properties. While computer-aided design of virtual molecules has made large progress, computer-assisted synthesis planning (CASP) to realize physical molecules is still in its infancy and lacks a performance level that would enable large-scale molecule discovery. CASP supports the search for multi-step synthesis routes, which is very challenging due to high branching factors in each synthesis step and the hidden rules that govern the reactions. The central and repeatedly applied step in CASP is reaction prediction, for which machine learning methods yield the best performance. We propose a novel reaction prediction approach that uses a deep learning architecture with modern Hopfield networks (MHNs) that is optimized by contrastive learning. An MHN is an associative memory that can store and retrieve chemical reactions in each layer of a deep learning architecture. We show that our MHN contrastive learning approach enables few- and zero-shot learning for reaction prediction which, in contrast to previous methods, can deal with rare, single, or even no training example(s) for a reaction. On a well established benchmark, our MHN approach pushes the state-of-the-art performance up by a large margin as it improves the predictive top-100 accuracy from $0.858\pm0.004$ to $0.959\pm0.004$. This advance might pave the way to large-scale molecule discovery.

Related papers

• BatGPT-Chem: A Foundation Large Model For Retrosynthesis Prediction [65.93303145891628]
  BatGPT-Chem is a large language model with 15 billion parameters, tailored for enhanced retrosynthesis prediction. Our model captures a broad spectrum of chemical knowledge, enabling precise prediction of reaction conditions. This development empowers chemists to adeptly address novel compounds, potentially expediting the innovation cycle in drug manufacturing and materials science.
  arXiv  Detail & Related papers  (2024-08-19T05:17:40Z)
• UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment [51.49238426241974]
  This paper introduces UAlign, a template-free graph-to-sequence pipeline for retrosynthesis prediction. By combining graph neural networks and Transformers, our method can more effectively leverage the inherent graph structure of molecules.
  arXiv  Detail & Related papers  (2024-03-25T03:23:03Z)
• Contextual Molecule Representation Learning from Chemical Reaction Knowledge [24.501564702095937]
  We introduce REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry. REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature.
  arXiv  Detail & Related papers  (2024-02-21T12:58:40Z)
• HD-Bind: Encoding of Molecular Structure with Low Precision, Hyperdimensional Binary Representations [3.3934198248179026]
  Hyperdimensional Computing (HDC) is a proposed learning paradigm that is able to leverage low-precision binary vector arithmetic. We show that HDC-based inference methods are as much as 90 times more efficient than more complex representative machine learning methods.
  arXiv  Detail & Related papers  (2023-03-27T21:21:46Z)
• Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction [53.06671763877109]
  We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
  arXiv  Detail & Related papers  (2023-02-04T01:32:40Z)
• Chemical-Reaction-Aware Molecule Representation Learning [88.79052749877334]
  We propose using chemical reactions to assist learning molecule representation. Our approach is proven effective to 1) keep the embedding space well-organized and 2) improve the generalization ability of molecule embeddings. Experimental results demonstrate that our method achieves state-of-the-art performance in a variety of downstream tasks.
  arXiv  Detail & Related papers  (2021-09-21T00:08:43Z)
• Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search [83.22850633478302]
  Retrosynthetic planning identifies a series of reactions that can lead to the synthesis of a target product. Existing methods either require expensive return estimation by rollout with high variance, or optimize for search speed rather than the quality. We propose Retro*, a neural-based A*-like algorithm that finds high-quality synthetic routes efficiently.
  arXiv  Detail & Related papers  (2020-06-29T05:53:33Z)
• Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning [75.95376096628135]
  We propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design. In this setup, the agent learns to navigate through the immense synthetically accessible chemical space. We describe how the end-to-end training in this study represents an important paradigm in radically expanding the synthesizable chemical space.
  arXiv  Detail & Related papers  (2020-04-26T21:40:03Z)
• Deep Molecular Programming: A Natural Implementation of Binary-Weight ReLU Neural Networks [7.700240949386079]
  We show how a BinaryConnect neural network trained in silico can be compiled to an equivalent chemical reaction network. Our work sets the stage for rich knowledge transfer between neural network and molecular programming communities.
  arXiv  Detail & Related papers  (2020-03-30T18:12:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.