Improving RNA Secondary Structure Design using Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2111.04504v1
• Date: Fri, 5 Nov 2021 02:54:06 GMT
• Title: Improving RNA Secondary Structure Design using Deep Reinforcement Learning
• Authors: Alexander Whatley, Zhekun Luo, Xiangru Tang
• Abstract summary: We propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches.
• Score: 69.63971634605797
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rising costs in recent years of developing new drugs and treatments have led to extensive research in optimization techniques in biomolecular design. Currently, the most widely used approach in biomolecular design is directed evolution, which is a greedy hill-climbing algorithm that simulates biological evolution. In this paper, we propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. In addition to experimenting with the vanilla implementations of each reinforcement learning algorithm from standard libraries, we analyze variants of each algorithm in which we modify the algorithm's reward function and tune the model's hyperparameters. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches and its ability to search the possible space of RNA sequences. We find that our DQN algorithm performs by far the best in this setting, contrasting with, in which PPO performs the best among all tested algorithms. Our results should be of interest to those in the biomolecular design community and should serve as a baseline for future experiments involving machine learning in molecule design.

Related papers

• Quantum-inspired Reinforcement Learning for Synthesizable Drug Design [20.00111975801053]
  We introduce a novel approach using the reinforcement learning method with quantum-inspired simulated annealing policy neural network to navigate the vast discrete space of chemical structures intelligently. Specifically, we employ a deterministic REINFORCE algorithm using policy neural networks to output transitional probability to guide state transitions and local search. Our methods are evaluated with the Practical Molecular Optimization (PMO) benchmark framework with a 10K query budget.
  arXiv  Detail & Related papers  (2024-09-13T20:43:16Z)
• Benchmark on Drug Target Interaction Modeling from a Structure Perspective [48.60648369785105]
  Drug-target interaction prediction is crucial to drug discovery and design. Recent methods, such as those based on graph neural networks (GNNs) and Transformers, demonstrate exceptional performance across various datasets. We conduct a comprehensive survey and benchmark for drug-target interaction modeling from a structure perspective, via integrating tens of explicit (i.e., GNN-based) and implicit (i.e., Transformer-based) structure learning algorithms.
  arXiv  Detail & Related papers  (2024-07-04T16:56:59Z)
• Tree Search-Based Evolutionary Bandits for Protein Sequence Optimization [44.356888079704156]
  Protein engineering is a daunting task due to the vast sequence space of any given protein. Protein engineering is typically conducted through an iterative process of adding mutations to the wild-type or lead sequences. We propose a tree search-based bandit learning method, which expands a tree starting from the initial sequence with the guidance of a bandit machine learning model.
  arXiv  Detail & Related papers  (2024-01-08T06:33:27Z)
• Utilizing Reinforcement Learning for de novo Drug Design [2.5740778707024305]
  We develop a unified framework for using reinforcement learning for de novo drug design. We study various on- and off-policy reinforcement learning algorithms and replay buffers to learn an RNN-based policy. Our findings suggest that it is advantageous to use at least both top-scoring and low-scoring molecules for updating the policy.
  arXiv  Detail & Related papers  (2023-03-30T07:40:50Z)
• Reinforced Genetic Algorithm for Structure-based Drug Design [38.134929249388406]
  Structure-based drug design (SBDD) aims to discover drug candidates by finding molecules that bind to a disease-related protein (targets) We propose Reinforced Genetic Algorithm (RGA) that uses neural models to prioritize the profitable design steps and suppress random-walk behavior.
  arXiv  Detail & Related papers  (2022-11-28T22:59:46Z)
• An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization [78.36413169647408]
  We study the effectiveness of various ZO optimization methods for optimizing molecular objectives. We show the advantages of ZO sign-based gradient descent (ZO-signGD) We demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite.
  arXiv  Detail & Related papers  (2022-10-27T01:58:10Z)
• Improved Algorithms for Neural Active Learning [74.89097665112621]
  We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting. We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
  arXiv  Detail & Related papers  (2022-10-02T05:03:38Z)
• Using Genetic Programming to Predict and Optimize Protein Function [65.25258357832584]
  We propose POET, a computational Genetic Programming tool based on evolutionary methods to enhance screening and mutagenesis in Directed Evolution. As a proof-of-concept we use peptides that generate MRI contrast detected by the Chemical Exchange Saturation Transfer mechanism. Our results indicate that a computational modelling tool like POET can help to find peptides with 400% better functionality than used before.
  arXiv  Detail & Related papers  (2022-02-08T18:08:08Z)
• JANUS: Parallel Tempered Genetic Algorithm Guided by Deep Neural Networks for Inverse Molecular Design [1.6114012813668934]
  Inverse molecular design, i.e., designing molecules with specific target properties, can be posed as an optimization problem. Janus is a genetic algorithm inspired by parallel tempering that propagates two populations, one for exploration and another for exploitation. Janus is augmented by a deep neural network that approximates molecular properties via active learning for enhanced sampling of the chemical space.
  arXiv  Detail & Related papers  (2021-06-07T23:41:34Z)
• AdaLead: A simple and robust adaptive greedy search algorithm for sequence design [55.41644538483948]
  We develop an easy-to-directed, scalable, and robust evolutionary greedy algorithm (AdaLead) AdaLead is a remarkably strong benchmark that out-competes more complex state of the art approaches in a variety of biologically motivated sequence design challenges.
  arXiv  Detail & Related papers  (2020-10-05T16:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.