ConRetroBert: EMA Stabilized Dual Encoders for Template-Based Single-Step Retrosynthesis

• URL: http://arxiv.org/abs/2605.12736v1
• Date: Tue, 12 May 2026 20:40:22 GMT
• Title: ConRetroBert: EMA Stabilized Dual Encoders for Template-Based Single-Step Retrosynthesis
• Abstract summary: Single step retrosynthesis predicts reactants by selecting and applying an explicit reaction template.<n>We present ConRetroBert, a dual encoder framework that reframes template based retrosynthesis as dense product template retrieval.
• Score: 2.152036255395242
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Template based single step retrosynthesis predicts reactants by selecting and applying an explicit reaction template, making each prediction traceable to a chemical transformation rule. This is useful for synthesis planning, but template based methods are often viewed as less competitive than template free models because template prediction is commonly formulated as global classification over a long tailed rule library. We argue that this weakness is not inherent to templates, but to the learning formulation. We present ConRetroBert, a dual encoder framework that reframes template based retrosynthesis as dense product template retrieval followed by candidate set listwise ranking. Stage 1 uses contrastive pretraining to learn a shared embedding space between products and reaction templates. Stage 2 refines template ranking over mined hard negative candidate sets with a multi positive listwise objective. To enable template side adaptation without destabilizing hard negative mining, ConRetroBert uses a slow moving exponential moving average template encoder for retrieval bank construction while updating the live template encoder through the ranking loss. On the local USPTO-50k benchmark, Stage 2 candidate set ranking improves top-1 reaction accuracy from 50.5% to 61.3%, while EMA stabilized template adaptation further improves it to 62.4%. Fine tuning from a leakage controlled USPTO-Full checkpoint reaches 75.4% top-1 accuracy on USPTO-50k. We also show that retrieval based template prediction is strong in the long tail of rare templates, and that many correct reactant predictions arise from alternative explicit templates rather than only the recorded positive label. Code and data are available at https://github.com/JahidBasher/ConRetroBert.

Related papers

• Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics [49.443264461057645]
  Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.<n>Progress in both paradigms is overwhelmingly tracked by generative perplexity (gen-PPL): the per-token negative log-likelihood of samples under a frozen autoregressive (AR) scorer.<n>We argue that this metric is unsound. By construction, gen-PPL measures only predictability under the scoring AR, not grammaticality or semantic coherence.
  arXiv  Detail & Related papers  (2026-06-07T02:35:56Z)
• RETROSPECT: RETROsynthesis via Sequential Prediction, and Chemically Transformed-ranking [0.004129405224409433]
  Single-step retrosynthesis needs both accurate first-ranked suggestions and candidate lists that are rich enough for downstream selection.<n>Our system, RETROSPECT, combines a single Transformer proposal model, which we call the ChemAlign Transformer, with a Lambda reranker.<n>The generator is trained with hybrid root-aligned and random SMILES augmentation, Pre-LayerNorm, tied embeddings, exponential moving average weights, and a differentiable atom-balance auxiliary loss.
  arXiv  Detail & Related papers  (2026-06-05T11:45:36Z)
• Decoupling Template Bias in CLIP: Harnessing Empty Prompts for Enhanced Few-Shot Learning [28.14553408545859]
  The Contrastive Language-Image Pre-Training (CLIP) model excels in few-shot learning by aligning visual and textual representations.<n>Our study shows that template-sample similarity (TSS), defined as the resemblance between a text template and an image sample, introduces bias.<n>This bias leads the model to rely on template proximity rather than true sample-to-category alignment.<n>We present a framework that uses empty prompts, textual inputs that convey the idea of "emptiness" without category information.
  arXiv  Detail & Related papers  (2025-12-09T13:51:05Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• A Template Is All You Meme [76.03172165923058]
  We create a knowledge base composed of more than 5,200 meme templates, information about them, and 54,000 examples of template instances.<n>To investigate the semantic signal of meme templates, we show that we can match memes in datasets to base templates contained in our knowledge base with a distance-based lookup.<n>Our examination of meme templates results in state-of-the-art performance for every dataset we consider, paving the way for analysis grounded in templateness.
  arXiv  Detail & Related papers  (2023-11-11T19:38:14Z)
• Retrosynthesis Prediction with Local Template Retrieval [112.23386062396622]
  Retrosynthesis, which predicts the reactants of a given target molecule, is an essential task for drug discovery. In this work, we introduce RetroKNN, a local reaction template retrieval method. We conduct comprehensive experiments on two widely used benchmarks, the USPTO-50K and USPTO-MIT.
  arXiv  Detail & Related papers  (2023-06-07T03:38:03Z)
• SemiRetro: Semi-template framework boosts deep retrosynthesis prediction [38.42917984016527]
  template-based (TB) and template-free (TF) molecule graph learning methods have shown promising results to retrosynthesis. We propose breaking a full-template into several semi-templates and embedding them into the two-step TF framework. Experimental results show that SemiRetro significantly outperforms both existing TB and TF methods.
  arXiv  Detail & Related papers  (2022-02-12T00:38:11Z)
• RetroComposer: Discovering Novel Reactions by Composing Templates for Retrosynthesis Prediction [63.14937611038264]
  We propose an innovative retrosynthesis prediction framework that can compose novel templates beyond training templates. Experimental results show that our method can produce novel templates for 328 test reactions in the USPTO-50K dataset.
  arXiv  Detail & Related papers  (2021-12-20T05:57:07Z)
• Template-free Prompt Tuning for Few-shot NER [46.59447116255979]
  We propose a more elegant method to reformulate NER tasks as LM problems without any templates. Specifically, we discard the template construction process while maintaining the word prediction paradigm of pre-training models. Experimental results demonstrate the effectiveness of the proposed method over bert-tagger and template-based method under few-shot setting.
  arXiv  Detail & Related papers  (2021-09-28T07:19:24Z)
• RetCL: A Selection-based Approach for Retrosynthesis via Contrastive Learning [107.64562550844146]
  Retrosynthesis is an emerging research area of deep learning. We propose a new approach that reformulating retrosynthesis into a selection problem of reactants from a candidate set of commercially available molecules. For learning the score functions, we also propose a novel contrastive training scheme with hard negative mining.
  arXiv  Detail & Related papers  (2021-05-03T12:47:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.