RetroOOD: Understanding Out-of-Distribution Generalization in Retrosynthesis Prediction

• URL: http://arxiv.org/abs/2312.10900v1
• Date: Mon, 18 Dec 2023 03:12:19 GMT
• Title: RetroOOD: Understanding Out-of-Distribution Generalization in Retrosynthesis Prediction
• Authors: Yemin Yu, Luotian Yuan, Ying Wei, Hanyu Gao, Xinhai Ye, Zhihua Wang, Fei Wu
• Abstract summary: Machine learning-assisted retrosynthesis prediction models have been gaining widespread adoption. Despite steady progress on standard benchmarks, our understanding of existing retrosynthesis prediction models under the premise of distribution shifts remains stagnant. We propose two model-agnostic techniques that can improve the OOD generalization of arbitrary off-the-shelf retrosynthesis prediction algorithms.
• Score: 15.699673606816496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning-assisted retrosynthesis prediction models have been gaining widespread adoption, though their performances oftentimes degrade significantly when deployed in real-world applications embracing out-of-distribution (OOD) molecules or reactions. Despite steady progress on standard benchmarks, our understanding of existing retrosynthesis prediction models under the premise of distribution shifts remains stagnant. To this end, we first formally sort out two types of distribution shifts in retrosynthesis prediction and construct two groups of benchmark datasets. Next, through comprehensive experiments, we systematically compare state-of-the-art retrosynthesis prediction models on the two groups of benchmarks, revealing the limitations of previous in-distribution evaluation and re-examining the advantages of each model. More remarkably, we are motivated by the above empirical insights to propose two model-agnostic techniques that can improve the OOD generalization of arbitrary off-the-shelf retrosynthesis prediction algorithms. Our preliminary experiments show their high potential with an average performance improvement of 4.6%, and the established benchmarks serve as a foothold for further retrosynthesis prediction research towards OOD generalization.

Related papers

• Reasoning-Driven Retrosynthesis Prediction with Large Language Models via Reinforcement Learning [16.284576756413184]
  RetroDFM-R is a reasoning-based large language model (LLM) designed specifically for chemical retrosynthesis.<n>It significantly enhances prediction accuracy and explainability, achieving a top-1 accuracy of 65.0% on the USPTO-50K benchmark.<n>It also accurately predicts multistep retrosynthetic routes reported in the literature for both real-world drug molecules and perovskite materials.
  arXiv  Detail & Related papers  (2025-07-23T12:13:06Z)
• Principled Out-of-Distribution Generalization via Simplicity [16.17883058788714]
  We study the compositional generalization abilities of diffusion models in image generation.<n>We develop a theoretical framework for OOD generalization via simplicity, quantified using a predefined simplicity metric.<n>We establish the first sharp sample complexity guarantees for learning the true, generalizable, simple model.
  arXiv  Detail & Related papers  (2025-05-28T17:44:10Z)
• Position: There are no Champions in Long-Term Time Series Forecasting [0.0]
  We focus on the need to shift focus away from pursuing ever-more complex models and towards enhancing benchmarking practices. To support our claim, we first perform a broad, thorough, and reproducible evaluation of the top-performing models on the most popular benchmark by training 3,500+ networks over 14 datasets.
  arXiv  Detail & Related papers  (2025-02-19T19:08:37Z)
• Challenging reaction prediction models to generalize to novel chemistry [12.33727805025678]
  We report a series of evaluations of a prototypical SMILES-based deep learning model. First, we illustrate how performance on randomly sampled datasets is overly optimistic compared to performance when generalizing to new patents or new authors. Second, we conduct time splits that evaluate how models perform when tested on reactions published in years after those in their training set, mimicking real-world deployment.
  arXiv  Detail & Related papers  (2025-01-11T23:49:14Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Retrosynthesis prediction enhanced by in-silico reaction data augmentation [66.5643280109899]
  We present RetroWISE, a framework that employs a base model inferred from real paired data to perform in-silico reaction generation and augmentation. On three benchmark datasets, RetroWISE achieves the best overall performance against state-of-the-art models.
  arXiv  Detail & Related papers  (2024-01-31T07:40:37Z)
• Forecast reconciliation for vaccine supply chain optimization [61.13962963550403]
  Vaccine supply chain optimization can benefit from hierarchical time series forecasting. Forecasts of different hierarchy levels become incoherent when higher levels do not match the sum of the lower levels forecasts. We tackle the vaccine sale forecasting problem by modeling sales data from GSK between 2010 and 2021 as a hierarchical time series.
  arXiv  Detail & Related papers  (2023-05-02T14:34:34Z)
• Better Modelling Out-of-Distribution Regression on Distributed Acoustic Sensor Data Using Anchored Hidden State Mixup [0.7455546102930911]
  Generalizing the application of machine learning models to situations where the statistical distribution of training and test data are different has been a complex problem. We introduce an anchored-based Out of Distribution (OOD) Regression Mixup algorithm, leveraging manifold hidden state mixup and observation similarities to form a novel regularization penalty. We demonstrate with an extensive evaluation the generalization performance of the proposed method against existing approaches, then show that our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-02-23T03:12:21Z)
• Feature-weighted Stacking for Nonseasonal Time Series Forecasts: A Case Study of the COVID-19 Epidemic Curves [0.0]
  We investigate ensembling techniques in forecasting and examine their potential for use in nonseasonal time-series. We propose using late data fusion, using a stacked ensemble of two forecasting models and two meta-features that prove their predictive power during a preliminary forecasting stage.
  arXiv  Detail & Related papers  (2021-08-19T14:44:46Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• Loss function based second-order Jensen inequality and its application to particle variational inference [112.58907653042317]
  Particle variational inference (PVI) uses an ensemble of models as an empirical approximation for the posterior distribution. PVI iteratively updates each model with a repulsion force to ensure the diversity of the optimized models. We derive a novel generalization error bound and show that it can be reduced by enhancing the diversity of models.
  arXiv  Detail & Related papers  (2021-06-09T12:13:51Z)
• Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future [73.03458424369657]
  In real-time forecasting in public health, data collection is a non-trivial and demanding task. 'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
  arXiv  Detail & Related papers  (2021-06-08T14:48:20Z)
• Double Robust Representation Learning for Counterfactual Prediction [68.78210173955001]
  We propose a novel scalable method to learn double-robust representations for counterfactual predictions. We make robust and efficient counterfactual predictions for both individual and average treatment effects. The algorithm shows competitive performance with the state-of-the-art on real world and synthetic data.
  arXiv  Detail & Related papers  (2020-10-15T16:39:26Z)
• Data Transfer Approaches to Improve Seq-to-Seq Retrosynthesis [1.6449390849183363]
  Retrosynthesis is a problem to infer reactant compounds to synthesize a given product compound through chemical reactions. Recent studies on retrosynthesis focus on proposing more sophisticated prediction models. The dataset to feed the models also plays an essential role in achieving the best generalizing models.
  arXiv  Detail & Related papers  (2020-10-02T05:27:51Z)
• Our Evaluation Metric Needs an Update to Encourage Generalization [24.6240575061124]
  Models that surpass human performance on popular benchmarks display significant degradation in performance on exposure to Out of Distribution data. We propose a simple and novel evaluation metric, WOOD Score, that encourages generalization during evaluation.
  arXiv  Detail & Related papers  (2020-07-14T08:15:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.