Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworks

• URL: http://arxiv.org/abs/2505.00530v1
• Date: Thu, 01 May 2025 13:57:20 GMT
• Title: Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworks
• Authors: Xinyu Wang, Jinbo Bi, Minghu Song,
• Abstract summary: We propose Partial SMILES Validation-PPO (PSV-PPO), a novel RL algorithm that incorporates real-time partial SMILES validation to prevent catastrophic forgetting.<n>PSV-PPO performs stepwise validation at each auto-regressive step, evaluating not only the selected token candidate but also all potential branches stemming from the prior partial sequence.<n>Our experiments on the PMO and GuacaMol benchmark datasets demonstrate that PSV-PPO significantly reduces the number of invalid generated structures.
• Score: 8.498666820216066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: SMILES-based molecule generation has emerged as a powerful approach in drug discovery. Deep reinforcement learning (RL) using large language model (LLM) has been incorporated into the molecule generation process to achieve high matching score in term of likelihood of desired molecule candidates. However, a critical challenge in this approach is catastrophic forgetting during the RL phase, where knowledge such as molecule validity, which often exceeds 99\% during pretraining, significantly deteriorates. Current RL algorithms applied in drug discovery, such as REINVENT, use prior models as anchors to retian pretraining knowledge, but these methods lack robust exploration mechanisms. To address these issues, we propose Partial SMILES Validation-PPO (PSV-PPO), a novel RL algorithm that incorporates real-time partial SMILES validation to prevent catastrophic forgetting while encouraging exploration. Unlike traditional RL approaches that validate molecule structures only after generating entire sequences, PSV-PPO performs stepwise validation at each auto-regressive step, evaluating not only the selected token candidate but also all potential branches stemming from the prior partial sequence. This enables early detection of invalid partial SMILES across all potential paths. As a result, PSV-PPO maintains high validity rates even during aggressive exploration of the vast chemical space. Our experiments on the PMO and GuacaMol benchmark datasets demonstrate that PSV-PPO significantly reduces the number of invalid generated structures while maintaining competitive exploration and optimization performance. While our work primarily focuses on maintaining validity, the framework of PSV-PPO can be extended in future research to incorporate additional forms of valuable domain knowledge, further enhancing reinforcement learning applications in drug discovery.

Related papers

• DrugR: Optimizing Molecular Drugs through LLM-based Explicit Reasoning [24.70952870676648]
  DrugR is a large language model that introduces explicit, step-by-step pharmacological reasoning into the optimization process.<n>Our approach integrates domain-specific continual pretraining, supervised fine-tuning via reverse data engineering, and self-balanced multi-granular reinforcement learning.<n> Experimental results demonstrate that DrugR achieves comprehensive enhancement across multiple properties without compromising structural similarity or target binding affinity.
  arXiv  Detail & Related papers  (2026-02-09T02:26:25Z)
• ReACT-Drug: Reaction-Template Guided Reinforcement Learning for de novo Drug Design [0.34155322317700576]
  We introduce bfReACT-Drug, a fully integrated, target-agnostic molecular design framework based on Reinforcement Learning.<n>This architecture highlights the potential of integrating structural biology, deep representation learning, and chemical rules to automate and accelerate rational drug design.
  arXiv  Detail & Related papers  (2025-12-24T05:29:35Z)
• Toward Routine CSP of Pharmaceuticals: A Fully Automated Protocol Using Neural Network Potentials [0.0]
  Crystal structure prediction (CSP) is a useful tool in pharmaceutical development for identifying and assessing risks associated with polymorphism.<n>Here, we introduce a fully automated, high- throughput CSP protocol designed to overcome these barriers.<n>The protocol's efficiency is driven by Lavo-NN, a novel neural network potential (NNP) architected and trained specifically for pharmaceutical crystal structure generation and ranking.
  arXiv  Detail & Related papers  (2025-07-22T04:26:16Z)
• Conditional Chemical Language Models are Versatile Tools in Drug Discovery [0.0]
  We present SAFE-T, a chemical modeling framework that conditions on biological context to prioritize molecules.<n>It supports principled scoring of molecules across tasks such as virtual screening, drug-target interaction prediction, and activity cliff detection.<n>It consistently achieves performance comparable to or better than existing approaches.
  arXiv  Detail & Related papers  (2025-07-14T13:42:39Z)
• De Novo Molecular Design Enabled by Direct Preference Optimization and Curriculum Learning [0.0]
  De novo molecular design has extensive applications in drug discovery and materials science.<n>The vast chemical space renders direct molecular searches computationally prohibitive, while traditional experimental screening is both time- and labor-intensive.<n>Direct Preference Optimization (DPO) from NLP uses molecular score-based sample pairs to maximize the likelihood difference between high- and low-quality molecules.
  arXiv  Detail & Related papers  (2025-04-02T06:00:21Z)
• Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation [6.495442425890008]
  Generative Flow Networks (GFlowNets) have recently emerged as a suitable framework for generating diverse and high-quality molecular structures.<n>In this work, we introduce Atomic GFlowNets (A-GFNs), a foundational generative model leveraging individual atoms as building blocks.<n>We propose an unsupervised pre-training approach using drug-like molecule datasets, which teaches A-GFNs about inexpensive yet informative molecular descriptors.
  arXiv  Detail & Related papers  (2025-03-08T20:41:07Z)
• Hierarchical Preference Optimization: Learning to achieve goals via feasible subgoals prediction [71.81851971324187]
  This work introduces Hierarchical Preference Optimization (HPO), a novel approach to hierarchical reinforcement learning (HRL) HPO addresses non-stationarity and infeasible subgoal generation issues when solving complex robotic control tasks. Experiments on challenging robotic navigation and manipulation tasks demonstrate impressive performance of HPO, where it shows an improvement of up to 35% over the baselines.
  arXiv  Detail & Related papers  (2024-11-01T04:58:40Z)
• VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment [66.80143024475635]
  We propose VinePPO, a straightforward approach to compute unbiased Monte Carlo-based estimates. We show that VinePPO consistently outperforms PPO and other RL-free baselines across MATH and GSM8K datasets.
  arXiv  Detail & Related papers  (2024-10-02T15:49:30Z)
• Regressor-free Molecule Generation to Support Drug Response Prediction [83.25894107956735]
  Conditional generation based on the target IC50 score can obtain a more effective sampling space. Regressor-free guidance combines a diffusion model's score estimation with a regression controller model's gradient based on number labels.
  arXiv  Detail & Related papers  (2024-05-23T13:22:17Z)
• Secrets of RLHF in Large Language Models Part I: PPO [81.01936993929127]
  Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. reinforcement learning with human feedback (RLHF) emerges as the pivotal technological paradigm underpinning this pursuit. In this report, we dissect the framework of RLHF, re-evaluate the inner workings of PPO, and explore how the parts comprising PPO algorithms impact policy agent training.
  arXiv  Detail & Related papers  (2023-07-11T01:55:24Z)
• A Systematic Survey of Chemical Pre-trained Models [38.57023440288189]
  Training Deep Neural Networks (DNNs) from scratch often requires abundant labeled molecules, which are expensive to acquire in the real world. To alleviate this issue, tremendous efforts have been devoted to Molecular Pre-trained Models (CPMs) CPMs are pre-trained using large-scale unlabeled molecular databases and then fine-tuned over specific downstream tasks.
  arXiv  Detail & Related papers  (2022-10-29T03:53:11Z)
• SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction [127.43571146741984]
  Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery. wet experiments remain the most reliable method, but they are time-consuming and resource-intensive. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. We present the SSM-DTA framework, which incorporates three simple yet highly effective strategies.
  arXiv  Detail & Related papers  (2022-06-20T14:53:25Z)
• Exploring Chemical Space with Score-based Out-of-distribution Generation [57.15855198512551]
  We propose a score-based diffusion scheme that incorporates out-of-distribution control in the generative differential equation (SDE) Since some novel molecules may not meet the basic requirements of real-world drugs, MOOD performs conditional generation by utilizing the gradients from a property predictor. We experimentally validate that MOOD is able to explore the chemical space beyond the training distribution, generating molecules that outscore ones found with existing methods, and even the top 0.01% of the original training pool.
  arXiv  Detail & Related papers  (2022-06-06T06:17:11Z)
• Generative Enriched Sequential Learning (ESL) Approach for Molecular Design via Augmented Domain Knowledge [1.4410716345002657]
  generative machine learning techniques can generate novel chemical structures based on molecular fingerprint representation. Lack of supervised domain knowledge can mislead the learning procedure to be relatively biased to the prevalent molecules observed in the training data. We alleviated this drawback by augmenting the training data with domain knowledge, e.g. quantitative estimates of the drug-likeness score (QEDs)
  arXiv  Detail & Related papers  (2022-04-05T20:16:11Z)
• Deep Learning for Virtual Screening: Five Reasons to Use ROC Cost Functions [80.12620331438052]
  deep learning has become an important tool for rapid screening of billions of molecules in silico for potential hits containing desired chemical features. Despite its importance, substantial challenges persist in training these models, such as severe class imbalance, high decision thresholds, and lack of ground truth labels in some datasets. We argue in favor of directly optimizing the receiver operating characteristic (ROC) in such cases, due to its robustness to class imbalance.
  arXiv  Detail & Related papers  (2020-06-25T08:46:37Z)
• Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning [75.95376096628135]
  We propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design. In this setup, the agent learns to navigate through the immense synthetically accessible chemical space. We describe how the end-to-end training in this study represents an important paradigm in radically expanding the synthesizable chemical space.
  arXiv  Detail & Related papers  (2020-04-26T21:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.