ChemRxivQuest: A Curated Chemistry Question-Answer Database Extracted from ChemRxiv Preprints

• URL: http://arxiv.org/abs/2505.05232v2
• Date: Fri, 13 Jun 2025 07:53:45 GMT
• Title: ChemRxivQuest: A Curated Chemistry Question-Answer Database Extracted from ChemRxiv Preprints
• Authors: Mahmoud Amiri, Thomas Bocklitz,
• Abstract summary: ChemRxivQuest is a curated dataset of 970 high-quality question-answer (QA) pairs from 155 ChemRxiv preprints across 17 subfields of chemistry.<n>Each QA pair is explicitly linked to its source text segment to ensure traceability and contextual accuracy.<n>ChemRxivQuest was constructed using an automated pipeline that combines optical character recognition (OCR), GPT-4o-based QA generation, and a fuzzy matching technique for answer verification.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid expansion of chemistry literature poses significant challenges for researchers seeking to efficiently access domain-specific knowledge. To support advancements in chemistry-focused natural language processing (NLP), we present ChemRxivQuest, a curated dataset of 970 high-quality question-answer (QA) pairs derived from 155 ChemRxiv preprints across 17 subfields of chemistry. Each QA pair is explicitly linked to its source text segment to ensure traceability and contextual accuracy. ChemRxivQuest was constructed using an automated pipeline that combines optical character recognition (OCR), GPT-4o-based QA generation, and a fuzzy matching technique for answer verification. The dataset emphasizes conceptual, mechanistic, applied, and experimental questions, enabling applications in retrieval-based QA systems, search engine development, and fine-tuning of domain-adapted large language models. We analyze the dataset's structure, coverage, and limitations, and outline future directions for expansion and expert validation. ChemRxivQuest provides a foundational resource for chemistry NLP research, education, and tool development.

Related papers

• QCBench: Evaluating Large Language Models on Domain-Specific Quantitative Chemistry [12.18966912295507]
  QCBench is a benchmark comprising 350 computational chemistry problems across 7 chemistry subfields.<n>Each problem focuses on pure calculations rooted in real-world chemical vertical fields.<n> Evaluations on 19 LLMs demonstrate a consistent performance degradation with increasing task complexity.
  arXiv  Detail & Related papers  (2025-08-03T08:55:42Z)
• A Multi-Agent System Enables Versatile Information Extraction from the Chemical Literature [8.306442315850878]
  We develop a multimodal large language model (MLLM)-based multi-agent system for robust and automated chemical information extraction.<n>Our system achieved an F1 score of 80.8% on a benchmark dataset of sophisticated multimodal chemical reaction graphics from the literature.
  arXiv  Detail & Related papers  (2025-07-27T11:16:57Z)
• ChemActor: Enhancing Automated Extraction of Chemical Synthesis Actions with LLM-Generated Data [53.78763789036172]
  We present ChemActor, a fully fine-tuned large language model (LLM) as a chemical executor to convert between unstructured experimental procedures and structured action sequences.<n>This framework integrates a data selection module that selects data based on distribution divergence, with a general-purpose LLM, to generate machine-executable actions from a single molecule input.<n>Experiments on reaction-to-description (R2D) and description-to-action (D2A) tasks demonstrate that ChemActor achieves state-of-the-art performance, outperforming the baseline model by 10%.
  arXiv  Detail & Related papers  (2025-06-30T05:11:19Z)
• Benchmarking Retrieval-Augmented Generation for Chemistry [28.592844362931853]
  Retrieval-augmented generation is a framework for enhancing large language models with external knowledge.<n>ChemRAG-Bench is a benchmark designed to assess the effectiveness of RAG across a diverse set of chemistry-related tasks.<n>ChemRAG-Toolkit is a modular toolkit that supports five retrieval algorithms and eight LLMs.
  arXiv  Detail & Related papers  (2025-05-12T15:34:45Z)
• ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning [64.2106664137118]
  ChemAgent is a novel framework designed to improve the performance of large language models (LLMs)<n>It is developed by decomposing chemical tasks into sub-tasks and compiling these sub-tasks into a structured collection that can be referenced for future queries.<n>When presented with a new problem, ChemAgent retrieves and refines pertinent information from the library, which we call memory.
  arXiv  Detail & Related papers  (2025-01-11T17:10:30Z)
• Large Language Models for Education: ChemTAsk -- An Open-Source Paradigm for Automated Q&A in the Graduate Classroom [1.5705429611931057]
  Large language models (LLMs) show promise for aiding graduate level education, but are limited by their training data and potential confabulations.<n>We developed ChemTAsk, an open-source pipeline that combines LLMs with retrieval-augmented generation (RAG) to provide accurate, context-specific assistance.
  arXiv  Detail & Related papers  (2025-01-10T02:19:47Z)
• ChemEval: A Comprehensive Multi-Level Chemical Evaluation for Large Language Models [62.37850540570268]
  Existing benchmarks in this domain fail to adequately meet the specific requirements of chemical research professionals. ChemEval identifies 4 crucial progressive levels in chemistry, assessing 12 dimensions of LLMs across 42 distinct chemical tasks. Results show that while general LLMs excel in literature understanding and instruction following, they fall short in tasks demanding advanced chemical knowledge.
  arXiv  Detail & Related papers  (2024-09-21T02:50:43Z)
• BatGPT-Chem: A Foundation Large Model For Retrosynthesis Prediction [65.93303145891628]
  BatGPT-Chem is a large language model with 15 billion parameters, tailored for enhanced retrosynthesis prediction. Our model captures a broad spectrum of chemical knowledge, enabling precise prediction of reaction conditions. This development empowers chemists to adeptly address novel compounds, potentially expediting the innovation cycle in drug manufacturing and materials science.
  arXiv  Detail & Related papers  (2024-08-19T05:17:40Z)
• ScholarChemQA: Unveiling the Power of Language Models in Chemical Research Question Answering [54.80411755871931]
  Question Answering (QA) effectively evaluates language models' reasoning and knowledge depth. Chemical QA plays a crucial role in both education and research by effectively translating complex chemical information into readily understandable format. This dataset reflects typical real-world challenges, including an imbalanced data distribution and a substantial amount of unlabeled data that can be potentially useful. We introduce a QAMatch model, specifically designed to effectively answer chemical questions by fully leveraging our collected data.
  arXiv  Detail & Related papers  (2024-07-24T01:46:55Z)
• Single and Multi-Hop Question-Answering Datasets for Reticular Chemistry with GPT-4-Turbo [0.5110571587151475]
  'RetChemQA' is a benchmark dataset designed to evaluate the capabilities of machine learning models in the domain of reticular chemistry. This dataset includes both single-hop and multi-hop question-answer pairs, encompassing approximately 45,000 Q&As for each type. The questions have been extracted from an extensive corpus of literature containing about 2,530 research papers from publishers including NAS, ACS, RSC, Elsevier, and Nature Publishing Group.
  arXiv  Detail & Related papers  (2024-05-03T14:29:54Z)
• An Autonomous Large Language Model Agent for Chemical Literature Data Mining [60.85177362167166]
  We introduce an end-to-end AI agent framework capable of high-fidelity extraction from extensive chemical literature. Our framework's efficacy is evaluated using accuracy, recall, and F1 score of reaction condition data.
  arXiv  Detail & Related papers  (2024-02-20T13:21:46Z)
• Chemist-X: Large Language Model-empowered Agent for Reaction Condition Recommendation in Chemical Synthesis [55.30328162764292]
  Chemist-X is a comprehensive AI agent that automates the reaction condition optimization (RCO) task in chemical synthesis.<n>The agent uses retrieval-augmented generation (RAG) technology and AI-controlled wet-lab experiment executions.<n>Results of our automatic wet-lab experiments, achieved by fully LLM-supervised end-to-end operation with no human in the lope, prove Chemist-X's ability in self-driving laboratories.
  arXiv  Detail & Related papers  (2023-11-16T01:21:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.