Quantum-inspired Reinforcement Learning for Synthesizable Drug Design

• URL: http://arxiv.org/abs/2409.09183v1
• Date: Fri, 13 Sep 2024 20:43:16 GMT
• Title: Quantum-inspired Reinforcement Learning for Synthesizable Drug Design
• Authors: Dannong Wang, Jintai Chen, Zhiding Liang, Tianfan Fu, Xiao-Yang Liu,
• Abstract summary: We introduce a novel approach using the reinforcement learning method with quantum-inspired simulated annealing policy neural network to navigate the vast discrete space of chemical structures intelligently. Specifically, we employ a deterministic REINFORCE algorithm using policy neural networks to output transitional probability to guide state transitions and local search. Our methods are evaluated with the Practical Molecular Optimization (PMO) benchmark framework with a 10K query budget.
• Score: 20.00111975801053
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Synthesizable molecular design (also known as synthesizable molecular optimization) is a fundamental problem in drug discovery, and involves designing novel molecular structures to improve their properties according to drug-relevant oracle functions (i.e., objective) while ensuring synthetic feasibility. However, existing methods are mostly based on random search. To address this issue, in this paper, we introduce a novel approach using the reinforcement learning method with quantum-inspired simulated annealing policy neural network to navigate the vast discrete space of chemical structures intelligently. Specifically, we employ a deterministic REINFORCE algorithm using policy neural networks to output transitional probability to guide state transitions and local search using genetic algorithm to refine solutions to a local optimum within each iteration. Our methods are evaluated with the Practical Molecular Optimization (PMO) benchmark framework with a 10K query budget. We further showcase the competitive performance of our method by comparing it against the state-of-the-art genetic algorithms-based method.

Related papers

• Dynamical Measure Transport and Neural PDE Solvers for Sampling [77.38204731939273]
  We tackle the task of sampling from a probability density as transporting a tractable density function to the target. We employ physics-informed neural networks (PINNs) to approximate the respective partial differential equations (PDEs) solutions. PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently.
  arXiv  Detail & Related papers  (2024-07-10T17:39:50Z)
• Genetic-guided GFlowNets for Sample Efficient Molecular Optimization [33.270494123656746]
  Recent advances in deep learning-based generative methods have shown promise but face the issue of sample efficiency. This paper proposes a novel algorithm for sample-efficient molecular optimization by distilling a powerful genetic algorithm into deep generative policy.
  arXiv  Detail & Related papers  (2024-02-05T04:12:40Z)
• Reinforcement Learning for Molecular Dynamics Optimization: A Stochastic Pontryagin Maximum Principle Approach [3.0077933778535706]
  We present a novel reinforcement learning framework designed to optimize molecular dynamics. Our framework focuses on the entire trajectory rather than just the final molecular configuration. Our method makes it suitable for applications in areas such as drug discovery and molecular design.
  arXiv  Detail & Related papers  (2022-12-06T20:44:24Z)
• An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization [78.36413169647408]
  We study the effectiveness of various ZO optimization methods for optimizing molecular objectives. We show the advantages of ZO sign-based gradient descent (ZO-signGD) We demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite.
  arXiv  Detail & Related papers  (2022-10-27T01:58:10Z)
• Neural Combinatorial Optimization: a New Player in the Field [69.23334811890919]
  This paper presents a critical analysis on the incorporation of algorithms based on neural networks into the classical optimization framework. A comprehensive study is carried out to analyse the fundamental aspects of such algorithms, including performance, transferability, computational cost and to larger-sized instances.
  arXiv  Detail & Related papers  (2022-05-03T07:54:56Z)
• Improving RNA Secondary Structure Design using Deep Reinforcement Learning [69.63971634605797]
  We propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches.
  arXiv  Detail & Related papers  (2021-11-05T02:54:06Z)
• Differentiable Scaffolding Tree for Molecular Optimization [47.447362691543304]
  We propose differentiable scaffolding tree (DST) that utilizes a learned knowledge network to convert discrete chemical structures to locally differentiable ones. Our empirical studies show the gradient-based molecular optimizations are both effective and sample efficient.
  arXiv  Detail & Related papers  (2021-09-22T01:16:22Z)
• JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design [1.6114012813668934]
  Inverse molecular design, i.e., designing molecules with specific target properties, can be posed as an optimization problem. Janus is a genetic algorithm inspired by parallel tempering that propagates two populations, one for exploration and another for exploitation. Janus is augmented by a deep neural network that approximates molecular properties via active learning for enhanced sampling of the chemical space.
  arXiv  Detail & Related papers  (2021-06-07T23:41:34Z)
• AdaLead: A simple and robust adaptive greedy search algorithm for sequence design [55.41644538483948]
  We develop an easy-to-directed, scalable, and robust evolutionary greedy algorithm (AdaLead) AdaLead is a remarkably strong benchmark that out-competes more complex state of the art approaches in a variety of biologically motivated sequence design challenges.
  arXiv  Detail & Related papers  (2020-10-05T16:40:38Z)
• Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning [0.0]
  We argue that existing generative methods are limited in their ability to favourably shift the distributions of molecular properties during optimization. We propose a novel Reinforcement Learning framework for molecular design in which an agent learns to directly optimize through a space of synthetically-accessible drug-like molecules.
  arXiv  Detail & Related papers  (2020-04-29T16:29:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.