MolRecBench-Wild: A Real-World Benchmark for Optical Chemical Structure Recognition

• URL: http://arxiv.org/abs/2605.05832v1
• Date: Thu, 07 May 2026 08:08:52 GMT
• Title: MolRecBench-Wild: A Real-World Benchmark for Optical Chemical Structure Recognition
• Abstract summary: Optical Chemical Structure Recognition aims to translate molecular diagrams in scientific literature into machine-readable formats.<n>We introduce MOSAIC, a dual-dimensional difficulty framework with 37 fine-grained labels that jointly characterize visual interference and chemical semantic challenges in molecular diagrams.<n>Based on this framework, we construct MolRecBench-Wild, a benchmark of 5,029 structures from 820 recent chemistry papers, covering the full difficulty spectrum observed in real publications.
• Score: 38.137828836708806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical Chemical Structure Recognition (OCSR) aims to translate molecular diagrams in scientific literature into machine-readable formats, but current systems remain unreliable on real-world images due to substantial visual and chemical complexity. We introduce MOSAIC, a dual-dimensional difficulty framework with 37 fine-grained labels that jointly characterize visual interference and chemical semantic challenges in molecular diagrams. Based on this framework, we construct MolRecBench-Wild, a benchmark of 5,029 structures from 820 recent chemistry papers, covering the full difficulty spectrum observed in real publications. To enable faithful semantic evaluation beyond SMILES and MolFile, we propose CARBON, a representation language capable of expressing valence variations, icon-based groups, and other non-standard chemical semantics. We further adopt a dual-track evaluation protocol supporting both CARBON and SMILES outputs for broad model compatibility. Comprehensive experiments over 18 OCSR-capable models reveal severe performance degradation on MolRecBench-Wild, exposing a large gap between previous patent benchmarks and real-world academic scenarios.

Related papers

• ChemDIRT: A Diversified Instruction, Representation, and Task Benchmark for Robust Chemistry-LLM Evaluation [18.06583682294473]
  ChemDIRT is a comprehensive evaluation framework designed to assess the robustness of chemical reasoning in large language models.<n>It measures performance across variations in instructions and molecular representations while spanning eight categories of chemistry tasks.<n>We benchmark a diverse set of open- and closed-source LLMs, revealing substantial prompt sensitivity, representation dependence, and uneven performance across task families.
  arXiv  Detail & Related papers  (2026-08-21T16:58:30Z)
• ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding [56.94419746470799]
  Large Language Models (LLMs) have revolutionized scientific text processing, but exhibit a significant capability gap when interpreting chemical reaction diagrams.<n>We propose the Chemical Visual Activation (ChemVA) framework, which employs a Visual Anchor mechanism to ground functional groups.<n>We evaluate our approach on OCRD-Bench, a newly constructed dataset featuring dense visual-semantic contexts.
  arXiv  Detail & Related papers  (2026-05-17T01:12:50Z)
• COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training [0.0]
  We introduce Minimum Risk Training to Optical chemical structure recognition (OCSR)<n> COMO is a closed-loop framework that mitigates exposure bias by directly optimizing over molecule-level, non-differentiable objectives.<n> Experiments on ten benchmarks including synthetic and real-world chemical diagrams demonstrate that COMO substantially outperforms existing rule-based and learning-based methods.
  arXiv  Detail & Related papers  (2026-04-26T05:48:21Z)
• MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learning [47.029225594084345]
  MolSight is a comprehensive learning framework that employs a three-stage training paradigm.<n>We show that MolSight achieves state-of-the-art performance in (stereo)chemical optical structure recognition.
  arXiv  Detail & Related papers  (2025-11-21T15:11:47Z)
• Benchmarking Multimodal LLMs on Recognition and Understanding over Chemical Tables [48.39080455781475]
  ChemTable is a large-scale benchmark of real-world chemical tables curated from the experimental sections of literature.<n>ChemTable includes expert-annotated cell polygons, logical layouts, and domain-specific labels, including reagents, catalysts, yields, and graphical components.<n>We evaluated a range of representative multimodal models, including both open-source and closed-source models, on ChemTable and reported a series of findings with practical and conceptual insights.
  arXiv  Detail & Related papers  (2025-06-13T00:45:41Z)
• GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition [60.76623665324548]
  GTR-Mol-VLM is a novel framework featuring two key innovations.<n>It emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions.<n>MolRec-Bench is the first benchmark designed for a fine-grained evaluation of graph-parsing accuracy in OCSR.
  arXiv  Detail & Related papers  (2025-06-09T08:47:10Z)
• MolMole: Molecule Mining from Scientific Literature [10.183226397580876]
  MolMole is a vision-based deep learning framework that unifies molecule detection, reaction diagram parsing, and optical chemical structure recognition.<n>We present a testset of 550 pages annotated with molecule bounding boxes, reaction labels, and MOLfiles.<n>Results demonstrate that MolMole outperforms existing toolkits on both our benchmark and public datasets.
  arXiv  Detail & Related papers  (2025-04-30T09:30:04Z)
• OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery [4.905515668299634]
  Optical Chemical Structure Understanding (OCSU) task aims to translate chemical structure diagrams into readable strings for both machine and chemist.<n>We propose DoubleCheck to enhance OCSR performance via attentive feature enhancement for local ambiguous atoms.<n>We also construct Vis-CheBI20, the first large-scale OCSU dataset.
  arXiv  Detail & Related papers  (2025-01-26T06:14:29Z)
• Building Open Knowledge Graph for Metal-Organic Frameworks (MOF-KG): Challenges and Case Studies [63.61566811532431]
  Metal-Organic Frameworks (MOFs) have great potential to revolutionize applications such as gas storage, molecular separations, chemical sensing, crystalline and drug delivery. The Cambridge Structural Database (CSD) reports 10,636 synthesized MOF crystals which in addition contains ca. 114,373 MOF-like structures. In this demo paper, we describe our effort on leveraging knowledge graph methods to facilitate MOF prediction, discovery, and synthesis.
  arXiv  Detail & Related papers  (2022-07-10T16:41:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.