TSSR: Two-Stage Swap-Reward-Driven Reinforcement Learning for Character-Level SMILES Generation

• URL: http://arxiv.org/abs/2601.04521v1
• Date: Thu, 08 Jan 2026 02:35:22 GMT
• Title: TSSR: Two-Stage Swap-Reward-Driven Reinforcement Learning for Character-Level SMILES Generation
• Authors: Jacob Ede Levine, Yun Lyan Luo, Sai Chandra Kosaraju,
• Abstract summary: TSSR is a Swap-Reward-driven reinforcement learning framework for character-level SMILES generation.<n>It rewards local token swaps that repair syntax, promoting transitions from invalid to parseable strings.<n>It converts a sparse terminal objective into a denser and more interpretable reward, improving both syntactic and chemical quality without reducing diversity.
• Score: 0.41998444721319217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The design of reliable, valid, and diverse molecules is fundamental to modern drug discovery, as improved molecular generation supports efficient exploration of the chemical space for potential drug candidates and reduces the cost of early design efforts. Despite these needs, current chemical language models that generate molecules as SMILES strings are vulnerable to compounding token errors: many samples are unparseable or chemically implausible, and hard constraints meant to prevent failure can restrict exploration. To address this gap, we introduce TSSR, a Two-Stage, Swap-Reward-driven reinforcement learning (RL) framework for character-level SMILES generation. Stage one rewards local token swaps that repair syntax, promoting transitions from invalid to parseable strings. Stage two provides chemistry-aware feedback from RDKit diagnostics, rewarding reductions in valence, aromaticity, and connectivity issues. The reward decomposes into interpretable terms (swap efficiency, error reduction, distance to validity), is model agnostic, and requires no task-specific labels or hand-crafted grammars. We evaluated TSSR on the MOSES benchmark using a GRU policy trained with PPO in both pure RL (P-RL) from random initialization and fine-tuning RL (F-RL) starting from a pretrained chemical language model, assessing 10,000 generated SMILES per run. In P-RL, TSSR significantly improves syntactic validity, chemical validity, and novelty. In F-RL, TSSR preserves drug-likeness and synthesizability while increasing validity and novelty. Token-level analysis shows that syntax edits and chemistry fixes act jointly to reduce RDKit detected errors. TSSR converts a sparse terminal objective into a denser and more interpretable reward, improving both syntactic and chemical quality without reducing diversity. TSSR is dataset-agnostic and can be adapted to various reinforcement learning approaches.

Related papers

• Error Amplification Limits ANN-to-SNN Conversion in Continuous Control [64.99656514469972]
  Spiking Neural Networks (SNNs) can achieve competitive performance by converting already existing well-trained Artificial Neural Networks (ANNs)<n>Existing conversion methods perform poorly in continuous control, where suitable baselines are largely absent.<n>We propose Cross-Step Residual Potential Initialization (CRPI), a lightweight training-free mechanism that carries over residual membrane potentials across decision steps to suppress temporally correlated errors.
  arXiv  Detail & Related papers  (2026-01-29T14:28:00Z)
• LANTERN: A Machine Learning Framework for Lipid Nanoparticle Transfection Efficiency Prediction [22.613971394957368]
  New ionizable lipids for efficient lipid nanoparticles (LNP)-mediated RNA delivery remains a critical bottleneck for RNA-based therapeutics development.<n>Recent advances have highlighted the potential of machine learning (ML) to predict transfection efficiency from molecular structure.<n>Here, we present LANTERN, a robust ML framework for predicting transfection efficiency based on ionizable lipid representation.
  arXiv  Detail & Related papers  (2025-07-03T22:49:49Z)
• Leveraging Partial SMILES Validation Scheme for Enhanced Drug Design in Reinforcement Learning Frameworks [8.498666820216066]
  We propose Partial SMILES Validation-PPO (PSV-PPO), a novel RL algorithm that incorporates real-time partial SMILES validation to prevent catastrophic forgetting.<n>PSV-PPO performs stepwise validation at each auto-regressive step, evaluating not only the selected token candidate but also all potential branches stemming from the prior partial sequence.<n>Our experiments on the PMO and GuacaMol benchmark datasets demonstrate that PSV-PPO significantly reduces the number of invalid generated structures.
  arXiv  Detail & Related papers  (2025-05-01T13:57:20Z)
• Unlocking Potential Binders: Multimodal Pretraining DEL-Fusion for Denoising DNA-Encoded Libraries [51.72836644350993]
  Multimodal Pretraining DEL-Fusion model (MPDF) We develop pretraining tasks applying contrastive objectives between different compound representations and their text descriptions. We propose a novel DEL-fusion framework that amalgamates compound information at the atomic, submolecular, and molecular levels.
  arXiv  Detail & Related papers  (2024-09-07T17:32:21Z)
• Text-Augmented Multimodal LLMs for Chemical Reaction Condition Recommendation [38.76977853056086]
  Chemma-RC is a text-augmented multimodal LLM to identify effective conditions through task-specific dialogue and condition generation.<n>Chemma-RC learns a unified representation of chemical reactions by aligning multiple modalities-including text corpus, reaction SMILES, and reaction graphs-within a shared embedding module.<n>Performance benchmarking on datasets showed high precision in identifying optimal conditions, with up to 17% improvement over the current state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-21T12:27:26Z)
• Learning Structurally Stabilized Representations for Multi-modal Lossless DNA Storage [32.00500955709341]
  Reed-Solomon coded single-stranded representation learning is a novel end-to-end model for learning representations for DNA storage. In contrast to existing learning-based methods, the proposed RSRL is inspired by both error-correction and structural biology. The experimental results obtained demonstrate that RSRL can store diverse types of data with much higher information density and durability but much lower error rates.
  arXiv  Detail & Related papers  (2024-07-17T06:31:49Z)
• Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scales [13.818149654692863]
  Reinforcement learning (RL) training is inherently unstable due to factors such as moving targets and high gradient variance.<n>In this work, we improve the stability of RL training by adapting the reverse cross entropy (RCE) from supervised learning for noisy data to define a symmetric RL loss.
  arXiv  Detail & Related papers  (2024-05-27T19:28:33Z)
• Contextual Molecule Representation Learning from Chemical Reaction Knowledge [24.501564702095937]
  We introduce REMO, a self-supervised learning framework that takes advantage of well-defined atom-combination rules in common chemistry. REMO pre-trains graph/Transformer encoders on 1.7 million known chemical reactions in the literature.
  arXiv  Detail & Related papers  (2024-02-21T12:58:40Z)
• Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint [104.53687944498155]
  Reinforcement learning (RL) has been widely used in training large language models (LLMs) We propose a new RL method named RLMEC that incorporates a generative model as the reward model. Based on the generative reward model, we design the token-level RL objective for training and an imitation-based regularization for stabilizing RL process.
  arXiv  Detail & Related papers  (2024-01-11T17:58:41Z)
• Generative error correction for code-switching speech recognition using large language models [49.06203730433107]
  Code-switching (CS) speech refers to the phenomenon of mixing two or more languages within the same sentence. We propose to leverage large language models (LLMs) and lists of hypotheses generated by an ASR to address the CS problem.
  arXiv  Detail & Related papers  (2023-10-17T14:49:48Z)
• A Simple Contrastive Learning Objective for Alleviating Neural Text Degeneration [56.64703901898937]
  We propose a new contrastive token learning objective that inherits the advantages of cross-entropy and unlikelihood training. Comprehensive experiments on language modeling and open-domain dialogue generation tasks show that the proposed contrastive token objective yields less repetitive texts.
  arXiv  Detail & Related papers  (2022-05-05T08:50:50Z)
• Deep Learning for Virtual Screening: Five Reasons to Use ROC Cost Functions [80.12620331438052]
  deep learning has become an important tool for rapid screening of billions of molecules in silico for potential hits containing desired chemical features. Despite its importance, substantial challenges persist in training these models, such as severe class imbalance, high decision thresholds, and lack of ground truth labels in some datasets. We argue in favor of directly optimizing the receiver operating characteristic (ROC) in such cases, due to its robustness to class imbalance.
  arXiv  Detail & Related papers  (2020-06-25T08:46:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.