Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad

• URL: http://arxiv.org/abs/2607.20935v1
• Date: Thu, 23 Jul 2026 05:30:45 GMT
• Title: Chemical Chain-of-Thought Functions as a Hallucination-Prone Molecular Scratchpad
• Abstract summary: Chemical reasoning language models are expected to derive molecular answers through faithful chain-of-thought (CoT)<n>Across four reasoning model families and twelve chemistry tasks, hallucination is widespread and largely decoupled from answer correctness.<n>This finding cautions against treating CoT as direct evidence of faithful reasoning and motivates process-level supervision beyond answer-only evaluation.
• Score: 32.236136102261945
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Chemical reasoning language models are expected to derive molecular answers through faithful chain-of-thought (CoT). However, across four reasoning model families and twelve chemistry tasks, hallucination is widespread and largely decoupled from answer correctness: correct answers often coexist with fabricated structural claims absent from the relevant molecules. Yet this does not make the reasoning trace computationally irrelevant. Attribution analyses suggest a shared scratchpad function expressed in model-specific forms: Chem-R and ether-0 rely on fragmented SMILES drafts, whereas ChemDFM-R emphasizes scaffold, positional, and naming cues. Notably, perturbing Chem-R's SMILES sketches degrades generation, showing that structural drafts can be causally load-bearing even when verbal structural claims are largely inert. Together, these results show that chemical CoT is neither a faithful explanation nor merely a post-hoc rationalization, but a hallucination-prone molecular scratchpad. This finding cautions against treating CoT as direct evidence of faithful reasoning and motivates process-level supervision beyond answer-only evaluation.

Related papers

• Accurate, Interdisciplinary and Transparent Structure-property Understanding with Deep Native Structural Reasoning [103.788281428856]
  We introduce SciReasoner, a scientific foundation model for native structural reasoning across proteins, small molecules and inorganic crystals.<n>SciReasoner discretizes coordinates, topologies and periodic connectivities into a unified structure-aware vocabulary.<n>Across 86 benchmarks, SciReasoner achieves state-of-the-art performance on 67 tasks.
  arXiv  Detail & Related papers  (2026-07-08T17:59:59Z)
• From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models [37.34302729762671]
  ChemCoTBench-V2 is a rule-verifiable diagnostic benchmark for evaluation of structured, verifier-addressable chemical reasoning traces.<n>It spans molecular understanding, molecule editing, molecular optimization, and reaction prediction, with 5,620 evaluation samples across 18 reporting tasks.
  arXiv  Detail & Related papers  (2026-06-02T13:47:19Z)
• Chem4DLLM: 4D Multimodal LLMs for Chemical Dynamics Understanding [52.499532583297885]
  ChemDU translates 4D molecular trajectories into interpretable natural-language explanations.<n>ChemDU focuses on fundamental dynamic scenarios, including gas-phase and catalytic reactions.<n>Chem4DBench is the first dataset pairing 4D molecular trajectories with expert-authored explanations.
  arXiv  Detail & Related papers  (2026-03-12T13:36:58Z)
• LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning [107.60117957760794]
  We introduce LatentChem, a latent reasoning interface that decouples chemical derivation from textual generation.<n>We show that LatentChem achieves a 59.88% non-tie win rate over strong CoT-based baselines on ChemCoTBench.<n>Our results provide empirical evidence that chemical reasoning is more naturally and effectively realized as continuous latent dynamics.
  arXiv  Detail & Related papers  (2026-02-06T01:28:27Z)
• Context-aware Graph Causality Inference for Few-Shot Molecular Property Prediction [8.42839603549236]
  We propose CaMol, a context-aware graph causality inference framework.<n>First, we introduce a context graph that encodes chemical knowledge by linking functional groups, molecules, and properties.<n>Second, we propose a learnable atom masking strategy to disentangle causal substructures from confounding ones.<n>Third, we introduce a distribution intervener that applies backdoor adjustment by combining causal substructures with chemically grounded confounders.
  arXiv  Detail & Related papers  (2026-01-16T09:49:50Z)
• How well can off-the-shelf LLMs elucidate molecular structures from mass spectra using chain-of-thought reasoning? [51.286853421822705]
  Large language models (LLMs) have shown promise for reasoning-intensive scientific tasks, but their capability for chemical interpretation is still unclear.<n>We introduce a Chain-of-Thought (CoT) prompting framework and benchmark that evaluate how LLMs reason about mass spectral data to predict molecular structures.<n>Our evaluation across metrics of SMILES validity, formula consistency, and structural similarity reveals that while LLMs can produce syntactically valid and partially plausible structures, they fail to achieve chemical accuracy or link reasoning to correct molecular predictions.
  arXiv  Detail & Related papers  (2026-01-09T20:08:42Z)
• Peeling Context from Cause for Multimodal Molecular Property Prediction [12.900046337617546]
  We introduce CLaP (Causal Layerwise Peeling), a framework that separates causal signal from context in a layerwise manner.<n>CLaP consistently improves MAE, MSE, and $R2$ over competitive baselines.<n>It produces atom-level causal saliency maps that highlight substructures responsible for predictions.
  arXiv  Detail & Related papers  (2025-11-10T04:29:33Z)
• Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Models [26.838674632817877]
  We develop a framework for explainable molecular property prediction based on language models, dubbed as Lamole. We take a string-based molecular representation -- Group SELFIES -- as input tokens to pretrain and fine-tune our Lamole. We show Lamole can achieve comparable classification accuracy and boost the explanation accuracy by up to 14.3%.
  arXiv  Detail & Related papers  (2024-05-25T03:27:04Z)
• Empirical Evidence for the Fragment level Understanding on Drug Molecular Structure of LLMs [16.508471997999496]
  We investigate whether and how language models understand the chemical spatial structure from 1D sequences. The results indicate that language models can understand chemical structures from the perspective of molecular fragments.
  arXiv  Detail & Related papers  (2024-01-15T12:53:58Z)
• Atom-Motif Contrastive Transformer for Molecular Property Prediction [68.85399466928976]
  Graph Transformer (GT) models have been widely used in the task of Molecular Property Prediction (MPP) We propose a novel Atom-Motif Contrastive Transformer (AMCT) which explores atom-level interactions and considers motif-level interactions. Our proposed AMCT is extensively evaluated on seven popular benchmark datasets, and both quantitative and qualitative results firmly demonstrate its effectiveness.
  arXiv  Detail & Related papers  (2023-10-11T10:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.