Applying Deep Reinforcement Learning to the HP Model for Protein Structure Prediction

• URL: http://arxiv.org/abs/2211.14939v1
• Date: Sun, 27 Nov 2022 21:17:48 GMT
• Title: Applying Deep Reinforcement Learning to the HP Model for Protein Structure Prediction
• Authors: Kaiyuan Yang, Houjing Huang, Olafs Vandans, Adithya Murali, Fujia Tian, Roland H.C. Yap, Liang Dai
• Abstract summary: In this work, we apply deep reinforcement learning to the HP model for protein folding. We find that a DQN based on long short-term memory (LSTM) architecture greatly enhances the RL learning ability and significantly improves the search process. Experimentally we show that it can find multiple distinct best-known solutions per trial.
• Score: 7.769624124148049
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A central problem in computational biophysics is protein structure prediction, i.e., finding the optimal folding of a given amino acid sequence. This problem has been studied in a classical abstract model, the HP model, where the protein is modeled as a sequence of H (hydrophobic) and P (polar) amino acids on a lattice. The objective is to find conformations maximizing H-H contacts. It is known that even in this reduced setting, the problem is intractable (NP-hard). In this work, we apply deep reinforcement learning (DRL) to the two-dimensional HP model. We can obtain the conformations of best known energies for benchmark HP sequences with lengths from 20 to 50. Our DRL is based on a deep Q-network (DQN). We find that a DQN based on long short-term memory (LSTM) architecture greatly enhances the RL learning ability and significantly improves the search process. DRL can sample the state space efficiently, without the need of manual heuristics. Experimentally we show that it can find multiple distinct best-known solutions per trial. This study demonstrates the effectiveness of deep reinforcement learning in the HP model for protein folding.

Related papers

• Hierarchical Preference Optimization: Learning to achieve goals via feasible subgoals prediction [71.81851971324187]
  This work introduces Hierarchical Preference Optimization (HPO), a novel approach to hierarchical reinforcement learning (HRL) HPO addresses non-stationarity and infeasible subgoal generation issues when solving complex robotic control tasks. Experiments on challenging robotic navigation and manipulation tasks demonstrate impressive performance of HPO, where it shows an improvement of up to 35% over the baselines.
  arXiv  Detail & Related papers  (2024-11-01T04:58:40Z)
• Reinforcement Learning for Sequence Design Leveraging Protein Language Models [14.477268882311991]
  We propose to use protein language models (PLMs) as a reward function to generate new sequences. We perform extensive experiments on various sequence lengths to benchmark RL-based approaches. We provide comprehensive evaluations along biological plausibility and diversity of the protein.
  arXiv  Detail & Related papers  (2024-07-03T14:31:36Z)
• Using quantum annealing to design lattice proteins [0.0]
  We demonstrate the fast and consistent identification of the correct HP model ground states using the D-Wave hybrid quantum-classical solver. An equally relevant biophysical challenge, called the protein design problem, is the inverse of the above. Here, we approach the design problem by a two-step procedure, implemented and executed on a D-Wave machine.
  arXiv  Detail & Related papers  (2024-02-14T10:28:43Z)
• Hyperparameter Sensitivity in Deep Outlier Detection: Analysis and a Scalable Hyper-Ensemble Solution [21.130842136324528]
  We conduct the first large-scale analysis on the HP sensitivity of deep OD methods. We design a HP-robust and scalable deep hyper-ensemble model called ROBOD that assembles models with varying HP configurations.
  arXiv  Detail & Related papers  (2022-06-15T16:46:00Z)
• Folding lattice proteins with quantum annealing [0.0]
  We develop a novel spin representation for lattice protein folding tailored for quantum annealing. With our encoding, the Hamiltonian by design has the quadratic structure required for calculations on an Ising-type annealer. Results are evaluated against existing exact results for HP chains with up to $N=30$ beads with 100% hit rate.
  arXiv  Detail & Related papers  (2022-05-12T13:30:57Z)
• Deep Reinforcement Learning with Spiking Q-learning [51.386945803485084]
  spiking neural networks (SNNs) are expected to realize artificial intelligence (AI) with less energy consumption. It provides a promising energy-efficient way for realistic control tasks by combining SNNs with deep reinforcement learning (RL)
  arXiv  Detail & Related papers  (2022-01-21T16:42:11Z)
• Improving RNA Secondary Structure Design using Deep Reinforcement Learning [69.63971634605797]
  We propose a new benchmark of applying reinforcement learning to RNA sequence design, in which the objective function is defined to be the free energy in the sequence's secondary structure. We show results of the ablation analysis that we do for these algorithms, as well as graphs indicating the algorithm's performance across batches.
  arXiv  Detail & Related papers  (2021-11-05T02:54:06Z)
• Genealogical Population-Based Training for Hyperparameter Optimization [1.0514231683620516]
  We experimentally demonstrate that our method cuts down by 2 to 3 times the computational cost required. Our method is search-algorithm so that the inner search routine can be any search algorithm like TPE, GP, CMA or random search.
  arXiv  Detail & Related papers  (2021-09-30T08:49:41Z)
• Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learning [56.17667147101263]
  In real-world tasks, reinforcement learning agents encounter situations that are not present during training time. To ensure reliable performance, the RL agents need to exhibit robustness against worst-case situations. We propose the Robust Hallucinated Upper-Confidence RL (RH-UCRL) algorithm to provably solve this problem.
  arXiv  Detail & Related papers  (2021-03-18T16:50:17Z)
• Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search [83.22850633478302]
  Retrosynthetic planning identifies a series of reactions that can lead to the synthesis of a target product. Existing methods either require expensive return estimation by rollout with high variance, or optimize for search speed rather than the quality. We propose Retro*, a neural-based A*-like algorithm that finds high-quality synthetic routes efficiently.
  arXiv  Detail & Related papers  (2020-06-29T05:53:33Z)
• Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planning [93.1435980666675]
  We show how optimistic exploration can be easily combined with state-of-the-art reinforcement learning algorithms. Our experiments demonstrate that optimistic exploration significantly speeds-up learning when there are penalties on actions.
  arXiv  Detail & Related papers  (2020-06-15T18:37:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.