MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition

• URL: http://arxiv.org/abs/2607.28532v1
• Date: Thu, 30 Jul 2026 17:03:05 GMT
• Title: MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition
• Abstract summary: Chemical structures appear in patents and the scientific literature as images.<n>For programmatic usage, such as indexing in databases or constructing machine learning model training sets, they must be transformed into line notations.<n>In this work, we treat both tasks as an image-to-text translation problem.<n>We propose OCSRGlyph, a state-of-the-art OCSR model, improving performance over prior methods by carefully considering stereochemistry.<n>For the Markush task, we introduce MarkushGlyph, a vision-language model that reads the entire Markush structure as an image.
• Score: 3.930866087901632
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Chemical structures appear in patents and the scientific literature as images. For programmatic usage, such as indexing in databases or constructing machine learning model training sets, they must be transformed into line notations. The two common forms of this task are translating an image of a single molecule (optical chemical structure recognition - OCSR) and translating a Markush structure that represents a family of molecules. While prior work in the former case is quite mature, Markush structure parsing remains a challenging task. In this work, we treat both tasks as an image-to-text translation problem. We then propose OCSRGlyph, a state-of-the-art OCSR model, improving performance over prior methods by carefully considering stereochemistry. For the Markush task, we introduce MarkushGlyph, a vision-language model that reads the entire Markush structure as an image. This contrasts with prior systems, which often use multiple stages to separately process visual and text input content. Finally, we introduce a new metric for determining the accuracy of Markush structure translations, handling failure modes present in prior metrics.

Related papers

• Bi-semantic Chemical Embedder for Joint Representation Learning of SMILES and Natural Language [2.9764168176721992]
  CheMatE is a chemistry-oriented embedding model that jointly captures molecular structure and domain-specific natural language.<n>We train the model using a novel, large-scale corpus of scientific documents that were constructed and curated from FineWeb and ChemPile.<n>This design exposes the model to SMILES-curated scientific literature, enabling bi-semantic understanding.
  arXiv  Detail & Related papers  (2026-08-04T15:57:38Z)
• MarkushGrapher-2: End-to-end Multimodal Recognition of Chemical Structures [2.7994101180516133]
  MarkushGrapher-2 is an end-to-end approach for the multimodal recognition of chemical structures in documents.<n>We introduce an automatic pipeline for constructing a large-scale dataset of real-world Markush structures.<n>Our approach substantially outperforms state-of-the-art models in multimodal Markush structure recognition.
  arXiv  Detail & Related papers  (2026-03-30T15:11:17Z)
• RTMol: Rethinking Molecule-text Alignment in a Round-trip View [4.597922051722059]
  We propose RTMol, a bidirectional alignment framework that unifies molecular captioning and text-to-SMILES generation through self-supervised round-trip learning.<n>Experiments demonstrate that RTMol enhances bidirectional alignment performance by up to 47% across various LLMs.
  arXiv  Detail & Related papers  (2025-11-15T09:55:55Z)
• RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioning [51.393018266721576]
  We propose the RxnCaption framework for the task of chemical Reaction Diagram Parsing (RxnDP)<n>Our framework reformulates the traditional coordinate prediction driven parsing process into an image captioning problem.<n>We introduce a strategy termed "BBox and Index as Visual Prompt" (BIVP), which uses our state-of-the-art molecular detector, MolYOLO, to pre-draw molecular bounding boxes and indices directly onto the input image.
  arXiv  Detail & Related papers  (2025-11-04T09:08:44Z)
• GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition [60.76623665324548]
  GTR-Mol-VLM is a novel framework featuring two key innovations.<n>It emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions.<n>MolRec-Bench is the first benchmark designed for a fine-grained evaluation of graph-parsing accuracy in OCSR.
  arXiv  Detail & Related papers  (2025-06-09T08:47:10Z)
• MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures [47.41884299076947]
  MarkushGrapher is a multi-modal approach for recognizing Markush structures in documents.<n>We propose a synthetic data generation pipeline that produces a wide range of realistic Markush structures.<n>M2S is the first annotated benchmark of real-world Markush structures.
  arXiv  Detail & Related papers  (2025-03-20T12:40:38Z)
• OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery [4.905515668299634]
  Optical Chemical Structure Understanding (OCSU) task aims to translate chemical structure diagrams into readable strings for both machine and chemist.<n>We propose DoubleCheck to enhance OCSR performance via attentive feature enhancement for local ambiguous atoms.<n>We also construct Vis-CheBI20, the first large-scale OCSU dataset.
  arXiv  Detail & Related papers  (2025-01-26T06:14:29Z)
• RFL: Simplifying Chemical Structure Recognition with Ring-Free Language [66.47173094346115]
  We propose a novel Ring-Free Language (RFL) to describe chemical structures in a hierarchical form.<n>RFL allows complex molecular structures to be decomposed into multiple parts, ensuring both uniqueness and conciseness.<n>We propose a universal Molecular Skeleton Decoder (MSD), which comprises a skeleton generation module that progressively predicts the molecular skeleton and individual rings.
  arXiv  Detail & Related papers  (2024-12-10T15:29:32Z)
• Story Visualization by Online Text Augmentation with Context Memory [64.86944645907771]
  We propose a novel memory architecture for the Bi-directional Transformer framework with an online text augmentation. The proposed method significantly outperforms the state of the arts in various metrics including FID, character F1, frame accuracy, BLEU-2/3, and R-precision.
  arXiv  Detail & Related papers  (2023-08-15T05:08:12Z)
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing [53.89917396428747]
  Self-supervised learning in vision-language processing exploits semantic alignment between imaging and text modalities. We explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model.
  arXiv  Detail & Related papers  (2023-01-11T16:35:33Z)
• Enhanced Modality Transition for Image Captioning [51.72997126838352]
  We build a Modality Transition Module (MTM) to transfer visual features into semantic representations before forwarding them to the language model. During the training phase, the modality transition network is optimised by the proposed modality loss. Experiments have been conducted on the MS-COCO dataset demonstrating the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2021-02-23T07:20:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.