Guiding Diffusion Models with Reinforcement Learning for Stable Molecule Generation

• URL: http://arxiv.org/abs/2508.16521v1
• Date: Fri, 22 Aug 2025 16:44:55 GMT
• Title: Guiding Diffusion Models with Reinforcement Learning for Stable Molecule Generation
• Authors: Zhijian Zhou, Junyi An, Zongkai Liu, Yunfei Shi, Xuan Zhang, Fenglei Cao, Chao Qu, Yuan Qi,
• Abstract summary: Reinforcement Learning with Physical Feedback (RLPF) is a novel framework that extends Denoising Diffusion Policy Optimization to 3D molecular generation.<n>RLPF introduces reward functions derived from force-field evaluations to guide the generation toward energetically stable and physically meaningful structures.<n> Experiments on the QM9 and GEOM-drug datasets demonstrate that RLPF significantly improves molecular stability compared to existing methods.
• Score: 16.01877423456416
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating physically realistic 3D molecular structures remains a core challenge in molecular generative modeling. While diffusion models equipped with equivariant neural networks have made progress in capturing molecular geometries, they often struggle to produce equilibrium structures that adhere to physical principles such as force field consistency. To bridge this gap, we propose Reinforcement Learning with Physical Feedback (RLPF), a novel framework that extends Denoising Diffusion Policy Optimization to 3D molecular generation. RLPF formulates the task as a Markov decision process and applies proximal policy optimization to fine-tune equivariant diffusion models. Crucially, RLPF introduces reward functions derived from force-field evaluations, providing direct physical feedback to guide the generation toward energetically stable and physically meaningful structures. Experiments on the QM9 and GEOM-drug datasets demonstrate that RLPF significantly improves molecular stability compared to existing methods. These results highlight the value of incorporating physics-based feedback into generative modeling. The code is available at: https://github.com/ZhijianZhou/RLPF/tree/verl_diffusion.

Related papers

• Uncertainty-Aware Multi-Objective Reinforcement Learning-Guided Diffusion Models for 3D De Novo Molecular Design [0.8749675983608171]
  We propose an uncertainty-aware Reinforcement Learning framework to guide the optimization of 3D molecular diffusion models.<n>Our results demonstrate the strong potential of RL-guided generative diffusion models for advancing automated molecular design.
  arXiv  Detail & Related papers  (2025-10-24T04:49:23Z)
• Energy-Based Flow Matching for Generating 3D Molecular Structure [7.055961224018033]
  We focus on flow matching and adopt an energy-based perspective to improve training and inference of structure generation models.<n>Our view results in a mapping function, that is directly learned to textititeratively map random configurations.<n>Experiments on protein docking and protein backbone generation consistently demonstrate the method's effectiveness.
  arXiv  Detail & Related papers  (2025-08-26T11:42:57Z)
• FlowMol3: Flow Matching for 3D De Novo Small-Molecule Generation [0.0]
  FlowMol3 is an open-source, multi-modal flow matching model that advances the state of the art for all-atom, small-molecule generation.<n>Our results highlight simple, transferable strategies for improving the stability and quality of diffusion- and flow-based molecular generative models.
  arXiv  Detail & Related papers  (2025-08-18T05:13:27Z)
• Torsional-GFN: a conditional conformation generator for small molecules [75.91029322687771]
  We introduce a conditional GFlowNet specifically designed to sample conformations of molecules proportionally to their Boltzmann distribution.<n>Our work presents a promising avenue for scaling the proposed approach to larger molecular systems.
  arXiv  Detail & Related papers  (2025-07-15T21:53:25Z)
• Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design [53.93023688824764]
  We address the problem of fine-tuning diffusion models for reward-guided generation in biomolecular design.<n>We propose an iterative distillation-based fine-tuning framework that enables diffusion models to optimize for arbitrary reward functions.<n>Our off-policy formulation, combined with KL divergence minimization, enhances training stability and sample efficiency compared to existing RL-based methods.
  arXiv  Detail & Related papers  (2025-07-01T05:55:28Z)
• Conditional Synthesis of 3D Molecules with Time Correction Sampler [58.0834973489875]
  Time-Aware Conditional Synthesis (TACS) is a novel approach to conditional generation on diffusion models. It integrates adaptively controlled plug-and-play "online" guidance into a diffusion model, driving samples toward the desired properties.
  arXiv  Detail & Related papers  (2024-11-01T12:59:25Z)
• Diffusion Models in $\textit{De Novo}$ Drug Design [0.0]
  Diffusion models have emerged as powerful tools for molecular generation, particularly in the context of 3D molecular structures. This review focuses on the technical implementation of diffusion models tailored for 3D molecular generation.
  arXiv  Detail & Related papers  (2024-06-07T06:34:13Z)
• AUTODIFF: Autoregressive Diffusion Modeling for Structure-based Drug Design [16.946648071157618]
  We propose a diffusion-based fragment-wise autoregressive generation model for structure-based drug design (SBDD) We design a novel molecule assembly strategy named conformal motif that preserves the conformation of local structures of molecules first. We then encode the interaction of the protein-ligand complex with an SE(3)-equivariant convolutional network and generate molecules motif-by-motif with diffusion modeling.
  arXiv  Detail & Related papers  (2024-04-02T14:44:02Z)
• Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting [53.44684898432997]
  Generation of molecules with desired chemical and biological properties is critical for drug discovery. We propose a probabilistic generative model to capture the joint distribution of molecules and their properties. Our method achieves very strong performances on various molecule design tasks.
  arXiv  Detail & Related papers  (2023-06-09T03:04:21Z)
• Geometric Latent Diffusion Models for 3D Molecule Generation [172.15028281732737]
  Generative models, especially diffusion models (DMs), have achieved promising results for generating feature-rich geometries. We propose a novel and principled method for 3D molecule generation named Geometric Latent Diffusion Models (GeoLDM)
  arXiv  Detail & Related papers  (2023-05-02T01:07:22Z)
• MUDiff: Unified Diffusion for Complete Molecule Generation [104.7021929437504]
  We present a new model for generating a comprehensive representation of molecules, including atom features, 2D discrete molecule structures, and 3D continuous molecule coordinates. We propose a novel graph transformer architecture to denoise the diffusion process. Our model is a promising approach for designing stable and diverse molecules and can be applied to a wide range of tasks in molecular modeling.
  arXiv  Detail & Related papers  (2023-04-28T04:25:57Z)
• Geometry-Complete Diffusion for 3D Molecule Generation and Optimization [3.8366697175402225]
  We introduce the Geometry-Complete Diffusion Model (GCDM) for 3D molecule generation. GCDM outperforms existing 3D molecular diffusion models by significant margins across conditional and unconditional settings. We also show that GCDM's geometric features can be repurposed to consistently optimize the geometry and chemical composition of existing 3D molecules.
  arXiv  Detail & Related papers  (2023-02-08T20:01:51Z)
• Learning Neural Generative Dynamics for Molecular Conformation Generation [89.03173504444415]
  We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph. We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
  arXiv  Detail & Related papers  (2021-02-20T03:17:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.