MolQuest: A Benchmark for Agentic Evaluation of Abductive Reasoning in Chemical Structure Elucidation

• URL: http://arxiv.org/abs/2603.25253v1
• Date: Thu, 26 Mar 2026 09:57:17 GMT
• Title: MolQuest: A Benchmark for Agentic Evaluation of Abductive Reasoning in Chemical Structure Elucidation
• Abstract summary: We introduce MolQuest, a novel agent-based evaluation framework for molecular structure elucidation.<n>Unlike existing datasets, MolQuest formalizes molecular structure elucidation as a multi-turn interactive task.<n>Results reveal that contemporary frontier models exhibit significant limitations in authentic scientific scenarios.
• Score: 20.62843594129358
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) hold considerable potential for advancing scientific discovery, yet systematic assessment of their dynamic reasoning in real-world research remains limited. Current scientific evaluation benchmarks predominantly rely on static, single-turn Question Answering (QA) formats, which are inadequate for measuring model performance in complex scientific tasks that require multi-step iteration and experimental interaction. To address this gap, we introduce MolQuest, a novel agent-based evaluation framework for molecular structure elucidation built upon authentic chemical experimental data. Unlike existing datasets, MolQuest formalizes molecular structure elucidation as a multi-turn interactive task, requiring models to proactively plan experimental steps, integrate heterogeneous spectral sources (e.g., NMR, MS), and iteratively refine structural hypotheses. This framework systematically evaluates LLMs' abductive reasoning and strategic decision-making abilities within a vast and complex chemical space. Empirical results reveal that contemporary frontier models exhibit significant limitations in authentic scientific scenarios: notably, even state-of-the-art (SOTA) models achieve an accuracy of only approximately 50%, while the performance of most other models remains below the 30% threshold. This work provides a reproducible and extensible framework for science-oriented LLM evaluation, our findings highlight the critical gap in current LLMs' strategic scientific reasoning, setting a clear direction for future research toward AI that can actively participate in the scientific process.

Related papers

• SciMIF: Understanding Multimodal Instruction Following in Scientific Domains [50.54744513908656]
  We introduce SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions.<n>Based on an analysis of 22 distinct tasks across 5 representative scientific disciplines, we propose a comprehensive taxonomy.<n>We develop a high-fidelity instruction injection pipeline to systematically augment existing scientific datasets.
  arXiv  Detail & Related papers  (2026-08-26T16:30:20Z)
• A Systematic Survey and Benchmark of Deep Learning for Molecular Property Prediction in the Foundation Model Era [57.5440098776759]
  Molecular property prediction integrates quantum chemistry, cheminformatics, and deep learning to connect molecular structure with biological behavior.<n>This survey traces four complementary paradigms, including Quantum, Descriptor Machine Learning, Geometric Deep Learning, and Foundation Models.<n> Benchmark analyses integrate evidence from both widely used datasets and datasets reflecting industry perspectives.
  arXiv  Detail & Related papers  (2026-04-17T15:16:33Z)
• Deciphering Scientific Reasoning Steps from Outcome Data for Molecule Optimization [22.850204283344524]
  We propose DESRO, a framework for deciphering scientific reasoning from outcomes.<n>By analyzing shared patterns and key differences within grouped data, a large language model (LLM) can recover the underlying logic.<n>We instantiate this framework in molecule optimization, a pivotal stage in drug discovery.
  arXiv  Detail & Related papers  (2026-03-13T06:25:27Z)
• Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis [51.83339196548892]
  ChemCraft is a novel framework that decouples chemical reasoning from knowledge storage.<n>ChemCraft achieves superior performance with minimal inference costs.<n>This work establishes a cost-effective and privacy-preserving paradigm for AI-aided chemistry.
  arXiv  Detail & Related papers  (2026-01-25T04:23:34Z)
• How well can off-the-shelf LLMs elucidate molecular structures from mass spectra using chain-of-thought reasoning? [51.286853421822705]
  Large language models (LLMs) have shown promise for reasoning-intensive scientific tasks, but their capability for chemical interpretation is still unclear.<n>We introduce a Chain-of-Thought (CoT) prompting framework and benchmark that evaluate how LLMs reason about mass spectral data to predict molecular structures.<n>Our evaluation across metrics of SMILES validity, formula consistency, and structural similarity reveals that while LLMs can produce syntactically valid and partially plausible structures, they fail to achieve chemical accuracy or link reasoning to correct molecular predictions.
  arXiv  Detail & Related papers  (2026-01-09T20:08:42Z)
• Evaluating Large Language Models in Scientific Discovery [91.732562776782]
  Large language models (LLMs) are increasingly applied to scientific research, yet prevailing science benchmarks probe decontextualized knowledge.<n>We introduce a scenario-grounded benchmark that evaluates LLMs across biology, chemistry, materials, and physics.<n>The framework assesses models at two levels: (i) question-level accuracy on scenario-tied items and (ii) project-level performance.
  arXiv  Detail & Related papers  (2025-12-17T16:20:03Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• The Evolving Role of Large Language Models in Scientific Innovation: Evaluator, Collaborator, and Scientist [3.7803247326675162]
  Scientific innovation is undergoing a paradigm shift driven by the rapid advancement of Large Language Models (LLMs)<n>This survey proposes a comprehensive framework to categorize the evolving roles of LLMs in scientific innovation across three hierarchical levels: Evaluator, Collaborator, and Scientist.
  arXiv  Detail & Related papers  (2025-07-16T00:11:01Z)
• Bridging the Plausibility-Validity Gap by Fine-Tuning a Reasoning-Enhanced LLM for Chemical Synthesis and Discovery [0.0]
  Large Language Models frequently generate outputs that appear scientifically reasonable yet violate fundamental principles.<n>We present a systematic approach combining a reasoning-centric model architecture with Low-Rank Adaptation fine-tuning.<n> Evaluation reveals substantial improvements: the fine-tuned system achieves 96.3% format adherence, 97.4% chemical validity, and 74.4% synthesis feasibility.
  arXiv  Detail & Related papers  (2025-07-09T23:05:23Z)
• RLDBF: Enhancing LLMs Via Reinforcement Learning With DataBase FeedBack [15.24890160206967]
  This study pioneers a systematic investigation into enhancing large language models with structured scientific data.<n>To address the inherent limitation of numerical insensitivity in large models, we propose an innovative methodology termed "Reinforcement Learning with Database Feedback"
  arXiv  Detail & Related papers  (2025-03-28T14:18:29Z)
• LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery [141.39722070734737]
  We propose to enhance the knowledge-driven, abstract reasoning abilities of Large Language Models with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework. We conduct experiments to demonstrate our framework's efficacy in law discovery and molecular design.
  arXiv  Detail & Related papers  (2024-05-16T03:04:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.