Uncertainty-aware reinforcement learning for chemical language models

• URL: http://arxiv.org/abs/2606.24990v1
• Date: Tue, 23 Jun 2026 14:42:46 GMT
• Title: Uncertainty-aware reinforcement learning for chemical language models
• Abstract summary: Reinforcement Learning (RL) has become a powerful paradigm for de novo molecular design.<n>We show that uncertainty-aware RL enables CLMs to explore chemical space more robustly.<n>This leads to more reliable hit discovery without compromising molecular score.
• Score: 0.5156484100374058
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement Learning (RL) has become a powerful paradigm for de novo molecular design, enabling Chemical Language Models (CLMs) to navigate and explore the chemical space while optimizing specific desired properties. However, the existing RL frameworks treat all scoring functions as deterministic oracles, neglecting the inherent uncertainty attached to the predictions of the different molecular properties. This can lead to the exploration of highly-uncertain regions of the chemical space, focusing on the generation of highly scored molecules which are poorly supported by the training data. This can destabilize the optimization process, yielding predictions that are far from their true values. We propose and compare two complementary ways of incorporating predictive uncertainty into RL. In the first one, uncertainty is treated as an additional optimization objective and incorporated along with the rest of the scoring functions, allowing the policy to trade off exploitation against reliability. Secondly, uncertainty is used to modulate policy updates, reducing the influence of molecules whose properties lie far outside the scoring function confidence domain. Both approaches were evaluated across three different settings: (i) a controlled model system, in which the prediction error is modeled as a Gaussian distribution, with a variance proportional to the distance to the training data; and two real-world tasks, making use of (ii) ChemProp models and (iii) a Conformal Prediction wrapper applied to a Random forest classifier. We show that uncertainty-aware RL enables CLMs to explore chemical space more robustly by favoring lower-uncertainty regions. This leads to more reliable hit discovery without compromising molecular score, increasing the true hit rate by 0.25 (from 0.5 to 0.75), and nearly doubling the total number of true hits.

Related papers

• CoMPASS: Collaborative Molecular Property Prediction via Adaptive Small-Large Model Synergy [8.012730274764019]
  Accurate molecular property prediction requires both statistical reliability and chemical reasoning.<n>CoMPASS is a retrieval-calibrated framework for small-large model collaboration.<n>Results suggest that generative reasoning should augment calibrated prediction through evidence-grounded, controlled corrections.
  arXiv  Detail & Related papers  (2026-08-31T12:14:01Z)
• Confidence is the key: how conformal prediction enhances the generative design of permeable peptides [0.8149107671670839]
  We present an RL-guided generative framework that designs permeable cyclic peptides using an uncertainty-aware permeability predictor.<n>To address predictive uncertainty, especially impacted by novel chemistry, we integrate conformal prediction (CP) as our uncertainty quantification method.
  arXiv  Detail & Related papers  (2026-05-07T07:01:45Z)
• Equivariant Evidential Deep Learning for Interatomic Potentials [55.6997213490859]
  Uncertainty quantification is critical for assessing the reliability of machine learning interatomic potentials in molecular dynamics simulations.<n>Existing UQ approaches for MLIPs are often limited by high computational cost or suboptimal performance.<n>We propose textitEquivariant Evidential Deep Learning for Interatomic Potentials ($texte2$IP), a backbone-agnostic framework that models atomic forces and their uncertainty jointly.
  arXiv  Detail & Related papers  (2026-02-11T02:00:25Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• Model Accuracy and Data Heterogeneity Shape Uncertainty Quantification in Machine Learning Interatomic Potentials [5.955636672018519]
  Machine learning interatomic potentials (MLIPs) enable accurate atomistic modelling, but reliable uncertainty quantification (UQ) remains elusive.<n>In this study, we investigate two UQ strategies, ensemble learning and D-optimality, within the atomic cluster expansion framework.
  arXiv  Detail & Related papers  (2025-08-05T12:52:49Z)
• Revisiting Essential and Nonessential Settings of Evidential Deep Learning [70.82728812001807]
  Evidential Deep Learning (EDL) is an emerging method for uncertainty estimation. We propose Re-EDL, a simplified yet more effective variant of EDL.
  arXiv  Detail & Related papers  (2024-10-01T04:27:07Z)
• YZS-model: A Predictive Model for Organic Drug Solubility Based on Graph Convolutional Networks and Transformer-Attention [9.018408514318631]
  Traditional methods often miss complex molecular structures, leading to inaccuracies. We introduce the YZS-Model, a deep learning framework integrating Graph Convolutional Networks (GCN), Transformer architectures, and Long Short-Term Memory (LSTM) networks. YZS-Model achieved an $R2$ of 0.59 and an RMSE of 0.57, outperforming benchmark models.
  arXiv  Detail & Related papers  (2024-06-27T12:40:29Z)
• Molecule Design by Latent Prompt Transformer [76.2112075557233]
  This work explores the challenging problem of molecule design by framing it as a conditional generative modeling task. We propose a novel generative model comprising three components: (1) a latent vector with a learnable prior distribution; (2) a molecule generation model based on a causal Transformer, which uses the latent vector as a prompt; and (3) a property prediction model that predicts a molecule's target properties and/or constraint values using the latent prompt.
  arXiv  Detail & Related papers  (2024-02-27T03:33:23Z)
• Uncertainty-biased molecular dynamics for learning uniformly accurate interatomic potentials [25.091146216183144]
  Active learning uses biased or unbiased molecular dynamics to generate candidate pools. Existing biased and unbiased MD-simulation methods are prone to miss either rare events or extrapolative regions. This work demonstrates that MD, when biased by the MLIP's energy uncertainty, simultaneously captures extrapolative regions and rare events.
  arXiv  Detail & Related papers  (2023-12-03T14:39:14Z)
• Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimation [51.809741427975105]
  This work aims to assess how well a model performs under distribution shifts without using labels. We use the nuclear norm that has been shown to be effective in characterizing both properties. We show that the nuclear norm is more accurate and robust in accuracy than existing methods.
  arXiv  Detail & Related papers  (2023-02-02T13:30:48Z)
• Optimizing Molecules using Efficient Queries from Property Evaluations [66.66290256377376]
  We propose QMO, a generic query-based molecule optimization framework. QMO improves the desired properties of an input molecule based on efficient queries. We show that QMO outperforms existing methods in the benchmark tasks of optimizing small organic molecules.
  arXiv  Detail & Related papers  (2020-11-03T18:51:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.