Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing

• URL: http://arxiv.org/abs/2603.15011v2
• Date: Tue, 17 Mar 2026 06:44:32 GMT
• Title: Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing
• Abstract summary: Reaction diagram parsing (RxnDP) is critical for extracting chemical synthesis information from literature.<n>Recent Vision-Language Models (VLMs) have emerged as a promising paradigm to automate this complex visual reasoning task.<n>This work enhances VLM-based RxnDP from two complementary perspectives: prompting representation and learning paradigms.
• Score: 52.825281124618535
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reaction diagram parsing (RxnDP) is critical for extracting chemical synthesis information from literature. Although recent Vision-Language Models (VLMs) have emerged as a promising paradigm to automate this complex visual reasoning task, their application is fundamentally bottlenecked by the inability to align visual chemical entities with pre-trained knowledge, alongside the inherent discrepancy between token-level training and reaction-level evaluation. To address these dual challenges, this work enhances VLM-based RxnDP from two complementary perspectives: prompting representation and learning paradigms. First, we propose Identifier as Visual Prompting (IdtVP), which leverages naturally occurring molecule identifiers (e.g., bold numerals like 1a) to activate the chemical knowledge acquired during VLM pre-training. IdtVP enables powerful zero-shot and out-of-distribution capabilities, outperforming existing prompting strategies. Second, to further optimize performance within fine-tuning paradigms, we introduce Re3-DAPO, a reinforcement learning algorithm that leverages verifiable rewards to directly optimize reaction-level metrics, thereby achieving consistent gains over standard supervised fine-tuning. Additionally, we release the ScannedRxn benchmark, comprising scanned historical reaction diagrams with real-world artifacts, to rigorously assess model robustness and out-of-distribution ability. Our contributions advance the accuracy and generalization of VLM-based reaction diagram parsing. We will release data, models, and code on GitHub.

Related papers

• ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding [56.94419746470799]
  Large Language Models (LLMs) have revolutionized scientific text processing, but exhibit a significant capability gap when interpreting chemical reaction diagrams.<n>We propose the Chemical Visual Activation (ChemVA) framework, which employs a Visual Anchor mechanism to ground functional groups.<n>We evaluate our approach on OCRD-Bench, a newly constructed dataset featuring dense visual-semantic contexts.
  arXiv  Detail & Related papers  (2026-05-17T01:12:50Z)
• ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding [18.366771283768344]
  We introduce ChemVLR, a chemical VLM designed to prioritize reasoning within the perception process.<n>Unlike conventional chemical VLMs, ChemVLR analyzes visual inputs in a fine-grained manner.<n>ChemVLR produces explicit and interpretable reasoning paths for complex visual chemical problems.
  arXiv  Detail & Related papers  (2026-04-08T05:01:59Z)
• DEX-AR: A Dynamic Explainability Method for Autoregressive Vision-Language Models [27.64151438258739]
  We present DEX-AR (Dynamic Explainability for AutoRegressive models), a novel explainability method.<n>It generates both per-token and sequence-level 2D heatmaps highlighting image regions crucial for the model's textual responses.<n>Our evaluation on ImageNet, VQAv2, and PascalVOC, shows a consistent improvement in both perturbation-based metrics.
  arXiv  Detail & Related papers  (2026-03-06T14:07:37Z)
• RxnNano:Training Compact LLMs for Chemical Reaction and Retrosynthesis Prediction via Hierarchical Curriculum Learning [20.904697361501174]
  We argue that the core challenge lies in instilling these knowledge into the models.<n>We propose a unified framework that prioritizes chemical understanding over scale through three key innovations.
  arXiv  Detail & Related papers  (2026-02-10T07:42:23Z)
• ViSS-R1: Self-Supervised Reinforcement Video Reasoning [84.1180294023835]
  We introduce a novel self-supervised reinforcement learning GRPO algorithm (Pretext-GRPO) within the standard R1 pipeline.<n>We also propose the ViSS-R1 framework, which streamlines and integrates pretext-task-based self-supervised learning directly into the MLLM's R1 post-training paradigm.
  arXiv  Detail & Related papers  (2025-11-17T07:00:42Z)
• RxnCaption: Reformulating Reaction Diagram Parsing as Visual Prompt Guided Captioning [51.393018266721576]
  We propose the RxnCaption framework for the task of chemical Reaction Diagram Parsing (RxnDP)<n>Our framework reformulates the traditional coordinate prediction driven parsing process into an image captioning problem.<n>We introduce a strategy termed "BBox and Index as Visual Prompt" (BIVP), which uses our state-of-the-art molecular detector, MolYOLO, to pre-draw molecular bounding boxes and indices directly onto the input image.
  arXiv  Detail & Related papers  (2025-11-04T09:08:44Z)
• GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition [60.76623665324548]
  GTR-Mol-VLM is a novel framework featuring two key innovations.<n>It emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions.<n>MolRec-Bench is the first benchmark designed for a fine-grained evaluation of graph-parsing accuracy in OCSR.
  arXiv  Detail & Related papers  (2025-06-09T08:47:10Z)
• Compile Scene Graphs with Reinforcement Learning [69.36723767339001]
  Next-token prediction is the fundamental principle for training large language models (LLMs)<n>We introduce R1-SGG, a multimodal LLM (M-LLM) initially trained via supervised fine-tuning (SFT) on the scene graph dataset.<n>We design a set of graph-centric rewards, including three recall-based variants -- Hard Recall, Hard Recall+Relax, and Soft Recall.
  arXiv  Detail & Related papers  (2025-04-18T10:46:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.