OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery

• URL: http://arxiv.org/abs/2501.15415v2
• Date: Thu, 22 May 2025 08:23:17 GMT
• Title: OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery
• Abstract summary: Optical Chemical Structure Understanding (OCSU) task aims to translate chemical structure diagrams into readable strings for both machine and chemist.<n>We propose DoubleCheck to enhance OCSR performance via attentive feature enhancement for local ambiguous atoms.<n>We also construct Vis-CheBI20, the first large-scale OCSU dataset.
• Score: 4.905515668299634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding the chemical structure from a graphical representation of a molecule is a challenging image caption task that would greatly benefit molecule-centric scientific discovery. Variations in molecular images and caption subtasks pose a significant challenge in both image representation learning and task modeling. Yet, existing methods only focus on a specific caption task that translates a molecular image into its graph structure, i.e., OCSR. In this paper, we propose the Optical Chemical Structure Understanding (OCSU) task, which extends low-level recognition to multilevel understanding and aims to translate chemical structure diagrams into readable strings for both machine and chemist. To facilitate the development of OCSU technology, we explore both OCSR-based and OCSR-free paradigms. We propose DoubleCheck to enhance OCSR performance via attentive feature enhancement for local ambiguous atoms. It can be cascaded with existing SMILES-based molecule understanding methods to achieve OCSU. Meanwhile, Mol-VL is a vision-language model end-to-end optimized for OCSU. We also construct Vis-CheBI20, the first large-scale OCSU dataset. Through comprehensive experiments, we demonstrate the proposed approaches excel at providing chemist-readable caption for chemical structure diagrams, which provide solid baselines for further research. Our code, model, and data are open-sourced at https://github.com/PharMolix/OCSU.

Related papers

• MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding [25.768840369692608]
  MolSight is a graph-aware vision-language model framework designed to enhance the understanding of molecular images.<n>Our experiments demonstrate that MolSight significantly outperforms existing VLMs, molecular LLMs, and specialized tools across multiple chemical visual understanding tasks.
  arXiv  Detail & Related papers  (2026-07-02T10:13:19Z)
• MolRecBench-Wild: A Real-World Benchmark for Optical Chemical Structure Recognition [38.137828836708806]
  Optical Chemical Structure Recognition aims to translate molecular diagrams in scientific literature into machine-readable formats.<n>We introduce MOSAIC, a dual-dimensional difficulty framework with 37 fine-grained labels that jointly characterize visual interference and chemical semantic challenges in molecular diagrams.<n>Based on this framework, we construct MolRecBench-Wild, a benchmark of 5,029 structures from 820 recent chemistry papers, covering the full difficulty spectrum observed in real publications.
  arXiv  Detail & Related papers  (2026-05-07T08:08:52Z)
• MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learning [47.029225594084345]
  MolSight is a comprehensive learning framework that employs a three-stage training paradigm.<n>We show that MolSight achieves state-of-the-art performance in (stereo)chemical optical structure recognition.
  arXiv  Detail & Related papers  (2025-11-21T15:11:47Z)
• $\ ext{M}^{2}$LLM: Multi-view Molecular Representation Learning with Large Language Models [59.125833618091846]
  We propose a multi-view framework that integrates three perspectives: the molecular structure view, the molecular task view, and the molecular rules view.<n>Experiments demonstrate that $textM2$LLM achieves state-of-the-art performance on multiple benchmarks across classification and regression tasks.
  arXiv  Detail & Related papers  (2025-08-12T05:46:47Z)
• CROP: Integrating Topological and Spatial Structures via Cross-View Prefixes for Molecular LLMs [10.16893890191528]
  We propose the CROss-view Prefixes (CROP) to enhance large language models' molecular understanding through efficient multi-view integration.<n>Our framework features a carefully designed SMILES Guided Resampler for view resampling, and a Structural Embedding Gate for converting the resulting embeddings into LLM's prefixes.
  arXiv  Detail & Related papers  (2025-08-09T10:06:28Z)
• GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition [60.76623665324548]
  GTR-Mol-VLM is a novel framework featuring two key innovations.<n>It emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions.<n>MolRec-Bench is the first benchmark designed for a fine-grained evaluation of graph-parsing accuracy in OCSR.
  arXiv  Detail & Related papers  (2025-06-09T08:47:10Z)
• Knowledge-aware contrastive heterogeneous molecular graph learning [77.94721384862699]
  We propose a paradigm shift by encoding molecular graphs into Heterogeneous Molecular Graph Learning (KCHML) KCHML conceptualizes molecules through three distinct graph views-molecular, elemental, and pharmacological-enhanced by heterogeneous molecular graphs and a dual message-passing mechanism. This design offers a comprehensive representation for property prediction, as well as for downstream tasks such as drug-drug interaction (DDI) prediction.
  arXiv  Detail & Related papers  (2025-02-17T11:53:58Z)
• RFL: Simplifying Chemical Structure Recognition with Ring-Free Language [66.47173094346115]
  We propose a novel Ring-Free Language (RFL) to describe chemical structures in a hierarchical form.<n>RFL allows complex molecular structures to be decomposed into multiple parts, ensuring both uniqueness and conciseness.<n>We propose a universal Molecular Skeleton Decoder (MSD), which comprises a skeleton generation module that progressively predicts the molecular skeleton and individual rings.
  arXiv  Detail & Related papers  (2024-12-10T15:29:32Z)
• FARM: Functional Group-Aware Representations for Small Molecules [55.281754551202326]
  We introduce Functional Group-Aware Representations for Small Molecules (FARM) FARM is a foundation model designed to bridge the gap between SMILES, natural language, and molecular graphs. We rigorously evaluate FARM on the MoleculeNet dataset, where it achieves state-of-the-art performance on 10 out of 12 tasks.
  arXiv  Detail & Related papers  (2024-10-02T23:04:58Z)
• MaskMol: Knowledge-guided Molecular Image Pre-Training Framework for Activity Cliffs [18.901322124389218]
  MaskMol is a knowledge-guided molecular image self-supervised learning framework. MaskMol accurately learns the representation of molecular images by considering multiple levels of molecular knowledge. Results demonstrate MaskMol's high accuracy and transferability in activity cliff estimation and compound potency prediction.
  arXiv  Detail & Related papers  (2024-09-02T03:03:22Z)
• SMiCRM: A Benchmark Dataset of Mechanistic Molecular Images [0.8192907805418583]
  We present a dataset designed to benchmark machine recognition capabilities of chemical molecules with arrow-pushing annotations. This dataset includes a machine-readable molecular identity for each image as well as mechanistic arrows showing electron flow during chemical reactions.
  arXiv  Detail & Related papers  (2024-07-25T18:52:10Z)
• UniIF: Unified Molecule Inverse Folding [67.60267592514381]
  We propose a unified model UniIF for inverse folding of all molecules. Our proposed method surpasses state-of-the-art methods on all tasks.
  arXiv  Detail & Related papers  (2024-05-29T10:26:16Z)
• Data-Efficient Molecular Generation with Hierarchical Textual Inversion [48.816943690420224]
  We introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method. HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution. Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution.
  arXiv  Detail & Related papers  (2024-05-05T08:35:23Z)
• Atom-Level Optical Chemical Structure Recognition with Limited Supervision [14.487346160322653]
  We propose a new chemical structure recognition tool that delivers state-of-the-art performance. Unlike previous approaches, our method provides atom-level localization. Our model is the first model to perform OCSR with atom-level entity detection with only SMILES supervision.
  arXiv  Detail & Related papers  (2024-04-02T09:01:21Z)
• MolNexTR: A Generalized Deep Learning Model for Molecular Image Recognition [4.510482519069965]
  MolNexTR is a novel image-to-graph deep learning model that collaborates to fuse the strengths of ConvNext and Vision-TRansformer. It can predict atoms and bonds simultaneously and understand their layout rules. In our test sets, MolNexTR has demonstrated superior performance, achieving an accuracy rate of 81-97%.
  arXiv  Detail & Related papers  (2024-03-06T13:17:41Z)
• Unveiling Molecular Moieties through Hierarchical Grad-CAM Graph Explainability [0.0]
  The integration of explainable methods to elucidate the specific contributions of molecular substructures to biological activity remains a significant challenge. We trained 20 GNN models on a dataset of small molecules with the goal of predicting their activity on 20 distinct protein targets from the Kinase family. We implemented the Hierarchical Grad-CAM graph Explainer framework, enabling an in-depth analysis of the molecular moieties driving protein-ligand binding stabilization.
  arXiv  Detail & Related papers  (2024-01-29T17:23:25Z)
• MultiModal-Learning for Predicting Molecular Properties: A Framework Based on Image and Graph Structures [2.5563339057415218]
  MolIG is a novel MultiModaL molecular pre-training framework for predicting molecular properties based on Image and Graph structures. It amalgamates the strengths of both molecular representation forms. It exhibits enhanced performance in downstream tasks pertaining to molecular property prediction within benchmark groups.
  arXiv  Detail & Related papers  (2023-11-28T10:28:35Z)
• MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter [91.77292826067465]
  Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception. We propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter.
  arXiv  Detail & Related papers  (2023-10-19T14:52:58Z)
• Bi-level Contrastive Learning for Knowledge-Enhanced Molecule Representations [68.32093648671496]
  We introduce GODE, which accounts for the dual-level structure inherent in molecules.<n> Molecules possess an intrinsic graph structure and simultaneously function as nodes within a broader molecular knowledge graph.<n>By pre-training two GNNs on different graph structures, GODE effectively fuses molecular structures with their corresponding knowledge graph substructures.
  arXiv  Detail & Related papers  (2023-06-02T15:49:45Z)
• MolScribe: Robust Molecular Structure Recognition with Image-To-Graph Generation [28.93523736883784]
  MolScribe is an image-to-graph model that explicitly predicts atoms and bonds, along with their geometric layouts, to construct the molecular structure. MolScribe significantly outperforms previous models, achieving 76-93% accuracy on public benchmarks.
  arXiv  Detail & Related papers  (2022-05-28T03:03:45Z)
• Molecular Contrastive Learning with Chemical Element Knowledge Graph [16.136921143416927]
  Molecular representation learning contributes to multiple downstream tasks such as molecular property prediction and drug design. We construct a Chemical Element Knowledge Graph (KG) to summarize microscopic associations between elements. The first module, knowledge-guided graph augmentation, augments the original molecular graph based on the Chemical Element KG. The second module, knowledge-aware graph representation, extracts molecular representations with a common graph encoder for the original molecular graph and a Knowledge-aware Message Passing Neural Network (KMPNN) to encode complex information in the augmented molecular graph.
  arXiv  Detail & Related papers  (2021-12-01T15:04:39Z)
• Chemical-Reaction-Aware Molecule Representation Learning [88.79052749877334]
  We propose using chemical reactions to assist learning molecule representation. Our approach is proven effective to 1) keep the embedding space well-organized and 2) improve the generalization ability of molecule embeddings. Experimental results demonstrate that our method achieves state-of-the-art performance in a variety of downstream tasks.
  arXiv  Detail & Related papers  (2021-09-21T00:08:43Z)
• IMG2SMI: Translating Molecular Structure Images to Simplified Molecular-input Line-entry System [29.946393284884778]
  We introduce IMG2SMI, a model which leverages Deep Residual Networks for image feature extraction and an encoder-decoder Transformer layers for molecule description generation. IMG2SMI outperforms OSRA-based systems by 163% in molecule similarity prediction as measured by the molecular MACCS Fingerprint Tanimoto Similarity. We also release a new molecule prediction dataset including 81 million molecules for molecule description generation.
  arXiv  Detail & Related papers  (2021-09-03T19:57:07Z)
• Self-Supervised Graph Transformer on Large-Scale Molecular Data [73.3448373618865]
  We propose a novel framework, GROVER, for molecular representation learning. GROVER can learn rich structural and semantic information of molecules from enormous unlabelled molecular data. We pre-train GROVER with 100 million parameters on 10 million unlabelled molecules -- the biggest GNN and the largest training dataset in molecular representation learning.
  arXiv  Detail & Related papers  (2020-06-18T08:37:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.