ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding

• URL: http://arxiv.org/abs/2605.17214v1
• Date: Sun, 17 May 2026 01:12:50 GMT
• Title: ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding
• Abstract summary: Large Language Models (LLMs) have revolutionized scientific text processing, but exhibit a significant capability gap when interpreting chemical reaction diagrams.<n>We propose the Chemical Visual Activation (ChemVA) framework, which employs a Visual Anchor mechanism to ground functional groups.<n>We evaluate our approach on OCRD-Bench, a newly constructed dataset featuring dense visual-semantic contexts.
• Score: 56.94419746470799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Large Language Models (LLMs) have revolutionized scientific text processing, they exhibit a significant capability gap when interpreting chemical reaction diagrams. We identify two fundamental bottlenecks restricting current systems: a Visual Deficit, where generic vision encoders struggle to resolve the strict topological connectivity of dense molecular graphs, and a Semantic Disconnect, where standard linear strings, such as SMILES, fail to effectively activate the model's latent chemical reasoning. To bridge these gaps, we propose the Chemical Visual Activation (ChemVA) framework, which employs a Visual Anchor mechanism to ground functional groups via hybrid-granularity detection, followed by a semantic alignment approach that translates visual features into entity names to maximize knowledge activation in LLMs. We evaluate our approach on OCRD-Bench, a newly constructed dataset featuring dense visual-semantic contexts and comprehensive reaction coverage to evaluate the full spectrum from recognition to reasoning. Extensive experiments on OCRD-Bench demonstrate that ChemVA achieves 92.0% structural recognition accuracy. By bridging visual and semantic bottlenecks, our framework delivers a consistent performance gain of approximately 20 percentage points across 9 diverse LLMs, enabling open-weight models to rival proprietary SOTA systems in complex chemical reasoning tasks.

Related papers

• MACReD: A Multi-Agent Collaborative Reasoning Framework for Reaction Diagram Parsing [54.118897112270425]
  We propose a hierarchical multi-agent framework that coordinates specialized agents for molecular perception, arrow understanding, text extraction, and reaction reconstruction within a unified VLM-guided architecture.<n>Experiments on the RxnScribe benchmark show that MACReD achieves state-of-the-art performance, with F1 scores of 75.2% and 84.6% under hard and soft match criteria.
  arXiv  Detail & Related papers  (2026-05-27T07:32:13Z)
• ChemVLR: Prioritizing Reasoning in Perception for Chemical Vision-Language Understanding [18.366771283768344]
  We introduce ChemVLR, a chemical VLM designed to prioritize reasoning within the perception process.<n>Unlike conventional chemical VLMs, ChemVLR analyzes visual inputs in a fine-grained manner.<n>ChemVLR produces explicit and interpretable reasoning paths for complex visual chemical problems.
  arXiv  Detail & Related papers  (2026-04-08T05:01:59Z)
• Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing [52.825281124618535]
  Reaction diagram parsing (RxnDP) is critical for extracting chemical synthesis information from literature.<n>Recent Vision-Language Models (VLMs) have emerged as a promising paradigm to automate this complex visual reasoning task.<n>This work enhances VLM-based RxnDP from two complementary perspectives: prompting representation and learning paradigms.
  arXiv  Detail & Related papers  (2026-03-16T09:17:05Z)
• Local-Global Multimodal Contrastive Learning for Molecular Property Prediction [0.4349640169711269]
  We propose LGM-CL, a local-global multimodal contrastive learning framework that jointly models molecular graphs and textual representations.<n>Experiments on MoleculeNet benchmarks demonstrate that LGM-CL achieves consistent and competitive performance across both classification and regression tasks.
  arXiv  Detail & Related papers  (2026-01-30T06:06:17Z)
• RxnBench: A Multimodal Benchmark for Evaluating Large Language Models on Chemical Reaction Understanding from Scientific Literature [25.978951548176706]
  We introduce RxnBench, a benchmark designed to rigorously evaluate MLLMs on chemical reaction understanding from scientific PDFs.<n> RxnBench comprises two tasks: Single-Figure QA (SF-QA), which tests fine-grained visual perception and mechanistic reasoning, and Full-Document QA (FD-QA), which challenges models to synthesize information from 108 articles.<n>Our evaluation of MLLMs reveals a critical capability gap: while models excel at extracting explicit text, they struggle with deep chemical logic and precise structural recognition.
  arXiv  Detail & Related papers  (2025-12-29T16:05:38Z)
• ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry [14.083820970280668]
  ChemVTS-Bench is a domain-authentic benchmark designed to evaluate the Visual-Textual-Symbolic (VTS) reasoning abilities of Multimodal Large Language Models (MLLMs)<n>ChemVTS-Bench contains diverse and challenging chemical problems spanning organic molecules, inorganic materials, and 3D crystal structures.<n>We develop an automated agent-based workflow that standardizes inference, verifies answers, and diagnoses failure modes.
  arXiv  Detail & Related papers  (2025-11-22T04:24:24Z)
• $\text{M}^{2}$LLM: Multi-view Molecular Representation Learning with Large Language Models [59.125833618091846]
  We propose a multi-view framework that integrates three perspectives: the molecular structure view, the molecular task view, and the molecular rules view.<n>Experiments demonstrate that $textM2$LLM achieves state-of-the-art performance on multiple benchmarks across classification and regression tasks.
  arXiv  Detail & Related papers  (2025-08-12T05:46:47Z)
• ChemActor: Enhancing Automated Extraction of Chemical Synthesis Actions with LLM-Generated Data [53.78763789036172]
  We present ChemActor, a fully fine-tuned large language model (LLM) as a chemical executor to convert between unstructured experimental procedures and structured action sequences.<n>This framework integrates a data selection module that selects data based on distribution divergence, with a general-purpose LLM, to generate machine-executable actions from a single molecule input.<n>Experiments on reaction-to-description (R2D) and description-to-action (D2A) tasks demonstrate that ChemActor achieves state-of-the-art performance, outperforming the baseline model by 10%.
  arXiv  Detail & Related papers  (2025-06-30T05:11:19Z)
• GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition [60.76623665324548]
  GTR-Mol-VLM is a novel framework featuring two key innovations.<n>It emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions.<n>MolRec-Bench is the first benchmark designed for a fine-grained evaluation of graph-parsing accuracy in OCSR.
  arXiv  Detail & Related papers  (2025-06-09T08:47:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.