URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment

• URL: http://arxiv.org/abs/2607.04688v1
• Date: Mon, 06 Jul 2026 05:32:09 GMT
• Title: URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment
• Abstract summary: This study introduces the URSA (Utilitarian RetroSynthesis Assessment) evaluation framework.<n>It provides the opportunity to benchmark the synthetic routes not only from a formal perspective, but also from a chemical plausibility perspective.<n>The study covers a comprehensive evaluation of both conventional end-to-end retrosynthesis solutions and LLMs for the synthesis planning task.
• Score: 3.6442441313881706
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Recent progress has produced both specialized deep-learning retrosynthesis systems and general-purpose large language models, but objective comparison remains difficult due to the lack of flexible, chemically interpretable benchmarking protocols. In the current study, we are introducing the URSA (Utilitarian RetroSynthesis Assessment) evaluation framework that provides the opportunity to benchmark the synthetic routes not only from a formal perspective, such as convergence to commercially available starting materials, but also from a chemical plausibility perspective, mimicking the way expert chemists evaluate the reactions and routes. The study covers a comprehensive evaluation of both conventional end-to-end retrosynthesis solutions and LLMs for the synthesis planning task on a set of novel, diverse target molecules with undisclosed synthetic routes, which represent realistic tasks in the daily drug design routine. We find that while LLMs can support high-level strategic planning, they currently underperform specialized retrosynthesis models in reliably solving synthesis planning tasks.

Related papers

• When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs [3.973137925060284]
  We propose a new benchmarking framework for single-step retrosynthesis.<n>By emphasizing plausibility over exact match, this approach better aligns with human synthesis planning practices.<n>We also introduce CREED, a novel dataset comprising millions of ChemCensor-validated reaction records for LLM training.
  arXiv  Detail & Related papers  (2026-02-03T14:03:32Z)
• Synthelite: Chemist-aligned and feasibility-aware synthesis planning with LLMs [3.7129661557601854]
  We introduce Synthelite, a synthesis planning framework that uses large language models to propose retrosynthetic transformations.<n> Synthelite can generate end-to-end synthesis routes by harnessing the intrinsic chemical knowledge and reasoning capabilities of LLMs.<n>Our experiments demonstrate that Synthelite can flexibly adapt its planning trajectory to diverse user-specified constraints, achieving up to 95% success rates.
  arXiv  Detail & Related papers  (2025-12-18T11:24:30Z)
• A Scientific Reasoning Model for Organic Synthesis Procedure Generation [12.609346156252393]
  We present QFANG, a scientific reasoning language model capable of generating precise, structured experimental procedures.<n>We introduce a Chemistry-Guided Reasoning (CGR) framework that produces chain-of-thought data grounded in chemical knowledge at scale.<n>We apply Reinforcement Learning from Verifiable Rewards (RLVR) to further enhance procedural accuracy.
  arXiv  Detail & Related papers  (2025-12-15T18:55:39Z)
• AOT*: Efficient Synthesis Planning via LLM-Empowered AND-OR Tree Search [22.026497456502806]
  AOT* is a framework that transforms retrosynthetic planning by integrating LLM-generated chemical synthesis pathways with systematic AND-OR tree search.<n>AOT* exhibits competitive solve rates using 3-5$times$ fewer iterations than existing LLM-based approaches.
  arXiv  Detail & Related papers  (2025-09-25T10:30:37Z)
• ChemActor: Enhancing Automated Extraction of Chemical Synthesis Actions with LLM-Generated Data [53.78763789036172]
  We present ChemActor, a fully fine-tuned large language model (LLM) as a chemical executor to convert between unstructured experimental procedures and structured action sequences.<n>This framework integrates a data selection module that selects data based on distribution divergence, with a general-purpose LLM, to generate machine-executable actions from a single molecule input.<n>Experiments on reaction-to-description (R2D) and description-to-action (D2A) tasks demonstrate that ChemActor achieves state-of-the-art performance, outperforming the baseline model by 10%.
  arXiv  Detail & Related papers  (2025-06-30T05:11:19Z)
• LLM-Augmented Chemical Synthesis and Design Decision Programs [18.41721617026997]
  We introduce an efficient scheme for encoding reaction pathways and present a new route-level search strategy.<n>We show that our LLM-augmented approach excels at retrosynthesis planning and extends naturally to the broader challenge of synthesizable molecular design.
  arXiv  Detail & Related papers  (2025-05-11T15:43:00Z)
• BatGPT-Chem: A Foundation Large Model For Retrosynthesis Prediction [65.93303145891628]
  BatGPT-Chem is a large language model with 15 billion parameters, tailored for enhanced retrosynthesis prediction. Our model captures a broad spectrum of chemical knowledge, enabling precise prediction of reaction conditions. This development empowers chemists to adeptly address novel compounds, potentially expediting the innovation cycle in drug manufacturing and materials science.
  arXiv  Detail & Related papers  (2024-08-19T05:17:40Z)
• ChemMiner: A Large Language Model Agent System for Chemical Literature Data Mining [56.15126714863963]
  ChemMiner is an end-to-end framework for extracting chemical data from literature.<n>ChemMiner incorporates three specialized agents: a text analysis agent for coreference mapping, a multimodal agent for non-textual information extraction, and a synthesis analysis agent for data generation.<n> Experimental results demonstrate reaction identification rates comparable to human chemists while significantly reducing processing time, with high accuracy, recall, and F1 scores.
  arXiv  Detail & Related papers  (2024-02-20T13:21:46Z)
• FusionRetro: Molecule Representation Fusion via In-Context Learning for Retrosynthetic Planning [58.47265392465442]
  Retrosynthetic planning aims to devise a complete multi-step synthetic route from starting materials to a target molecule. Current strategies use a decoupled approach of single-step retrosynthesis models and search algorithms. We propose a novel framework that utilizes context information for improved retrosynthetic planning.
  arXiv  Detail & Related papers  (2022-09-30T08:44:58Z)
• RetCL: A Selection-based Approach for Retrosynthesis via Contrastive Learning [107.64562550844146]
  Retrosynthesis is an emerging research area of deep learning. We propose a new approach that reformulating retrosynthesis into a selection problem of reactants from a candidate set of commercially available molecules. For learning the score functions, we also propose a novel contrastive training scheme with hard negative mining.
  arXiv  Detail & Related papers  (2021-05-03T12:47:57Z)
• Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning [75.95376096628135]
  We propose a novel forward synthesis framework powered by reinforcement learning (RL) for de novo drug design. In this setup, the agent learns to navigate through the immense synthetically accessible chemical space. We describe how the end-to-end training in this study represents an important paradigm in radically expanding the synthesizable chemical space.
  arXiv  Detail & Related papers  (2020-04-26T21:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.