Adopting Reinforcement Learning with Verifiable Rewards for Molecular Generation

• URL: http://arxiv.org/abs/2607.19044v1
• Date: Tue, 21 Jul 2026 12:34:59 GMT
• Title: Adopting Reinforcement Learning with Verifiable Rewards for Molecular Generation
• Abstract summary: Leveraging large language models for molecular generation has shown remarkable potential in chemical and drug design.<n>We propose textbfLLMol, a principled reinforcement learning framework that directly incorporates verifiable rewards for targeted molecule generation.
• Score: 17.176795343295854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Leveraging large language models (LLMs) for molecular generation has shown remarkable potential in chemical and drug design. Current methods primarily rely on supervised training or fine-tuning with limited datasets, which are insufficient to capture complex molecular design objectives. While some approaches attempt to guide generation toward specific goals, they often lack direct optimization mechanisms, making it difficult to align generated molecules with desired properties. To tackle these challenges, we propose \textbf{LLMol}, a principled reinforcement learning framework that directly incorporates verifiable rewards for targeted molecule generation. The key insight is to formulate molecular design as a goal-conditioned sequence prediction task, where verifiable rewards serve as explicit supervision to drive generation toward desired objectives. LLMol follows a two-stage training paradigm combining supervised learning and reinforcement learning. In the first stage, large language models are supervised fine-tuned to capture chemical syntax and molecular distributions. In the second stage, we introduce Reinforcement Learning with Verifiable Rewards (RLVR), which directly integrates property-based reward signals to guide molecular generation toward task-specific objectives. To address the high variance and instability common in discrete sequence optimization, we adopt Group Relative Policy Optimization (GRPO), a stable on-policy algorithm that smooths reward signals and improves training robustness. This framework enables LLMol to effectively handle a range of molecular design tasks, including single-property targeting (e.g., penalized logP, QED) and structure-constrained optimization. Experimental results demonstrate that LLMol consistently outperforms existing methods, achieving higher success rates and improved efficiency across diverse molecular benchmarks.

Related papers

• Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization [85.4863852505905]
  We introduce unsupervised reward optimization of protein language models (PLMs)<n>Our key insight is that task-agnostic rewards, which combine intrinsic model uncertainty with semantic consistency informed by protein representation models, exhibit strong correlation with controllability measures across base models and temperature regimes.<n>Our framework provides a scalable pathway toward controllable biomolecular design in settings where labeled preferences or experimental feedback are scarce or unavailable.
  arXiv  Detail & Related papers  (2026-06-17T11:42:01Z)
• MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models [48.34354151490484]
  We introduce MolRGen, a benchmark and dataset for training and evaluating reasoning-based large language models.<n>First, we propose a setting to evaluate and train models for textitde novo molecular generation and property prediction.<n>Second, we introduce a novel diversity-aware top-$k$ score that captures both the quality and diversity of generated molecules.
  arXiv  Detail & Related papers  (2026-03-18T20:21:04Z)
• Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoning [58.644854860003704]
  Large language models (LLMs) benefit from supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR) in reasoning tasks.<n>We show that answer-only SFT on the reference molecules collapses reasoning, and RLVR provides sparse feedback under similarity constraints.<n>We introduce Reference-guided Policy Optimization (RePO), an optimization approach that learns from reference molecules without requiring trajectory data.
  arXiv  Detail & Related papers  (2026-03-06T04:39:08Z)
• DrugR: Optimizing Molecular Drugs through LLM-based Explicit Reasoning [24.70952870676648]
  DrugR is a large language model that introduces explicit, step-by-step pharmacological reasoning into the optimization process.<n>Our approach integrates domain-specific continual pretraining, supervised fine-tuning via reverse data engineering, and self-balanced multi-granular reinforcement learning.<n> Experimental results demonstrate that DrugR achieves comprehensive enhancement across multiple properties without compromising structural similarity or target binding affinity.
  arXiv  Detail & Related papers  (2026-02-09T02:26:25Z)
• Empowering LLMs for Structure-Based Drug Design via Exploration-Augmented Latent Inference [5.052013621974765]
  Large Language Models (LLMs) possess strong representation and reasoning capabilities, but their application to structure-based drug design (SBDD) is limited by insufficient understanding of protein structures and unpredictable molecular generation.<n>We propose Exploration-Augmented Latent Inference for LLMs (ELILLM), a framework that reinterprets the LLM generation process as an encoding, latent space exploration, and decoding workflow.<n>ELILLM explicitly explores portions of the design problem beyond the model's current knowledge while using a decoding module to handle familiar regions, generating chemically valid and synthetically reasonable molecules.
  arXiv  Detail & Related papers  (2026-01-20T08:10:48Z)
• Reasoning-Enhanced Large Language Models for Molecular Property Prediction [19.593493317167646]
  Molecular property prediction is crucial for drug discovery and materials science.<n>Existing approaches suffer from limited interpretability, poor cross-task generalization, and lack of chemical reasoning capabilities.<n>We propose MPPReasoner, a multimodal large language model that incorporates chemical reasoning for molecular property prediction.
  arXiv  Detail & Related papers  (2025-10-11T15:05:45Z)
• Many-Shot In-Context Learning for Molecular Inverse Design [56.65345962071059]
  Large Language Models (LLMs) have demonstrated great performance in few-shot In-Context Learning (ICL) We develop a new semi-supervised learning method that overcomes the lack of experimental data available for many-shot ICL. As we show, the new method greatly improves upon existing ICL methods for molecular design while being accessible and easy to use for scientists.
  arXiv  Detail & Related papers  (2024-07-26T21:10:50Z)
• DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization [49.85944390503957]
  DecompOpt is a structure-based molecular optimization method based on a controllable and diffusion model. We show that DecompOpt can efficiently generate molecules with improved properties than strong de novo baselines.
  arXiv  Detail & Related papers  (2024-03-07T02:53:40Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Retrieval-based Controllable Molecule Generation [63.44583084888342]
  We propose a new retrieval-based framework for controllable molecule generation. We use a small set of molecules to steer the pre-trained generative model towards synthesizing molecules that satisfy the given design criteria. Our approach is agnostic to the choice of generative models and requires no task-specific fine-tuning.
  arXiv  Detail & Related papers  (2022-08-23T17:01:16Z)
• Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning [0.0]
  We argue that existing generative methods are limited in their ability to favourably shift the distributions of molecular properties during optimization. We propose a novel Reinforcement Learning framework for molecular design in which an agent learns to directly optimize through a space of synthetically-accessible drug-like molecules.
  arXiv  Detail & Related papers  (2020-04-29T16:29:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.