ChemPile: A 250GB Diverse and Curated Dataset for Chemical Foundation Models

• URL: http://arxiv.org/abs/2505.12534v1
• Date: Sun, 18 May 2025 20:22:21 GMT
• Title: ChemPile: A 250GB Diverse and Curated Dataset for Chemical Foundation Models
• Authors: Adrian Mirza, Nawaf Alampara, Martiño Ríos-García, Mohamed Abdelalim, Jack Butler, Bethany Connolly, Tunca Dogan, Marianna Nezhurina, Bünyamin Şen, Santosh Tirunagari, Mark Worrall, Adamo Young, Philippe Schwaller, Michael Pieler, Kevin Maik Jablonka,
• Abstract summary: We present the ChemPile, an open dataset containing over 75 billion tokens of curated chemical data.<n>The dataset mirrors the human learning journey through chemistry.<n>ChemPile captures both foundational concepts and domain-specific complexity.
• Score: 2.0815739337757555
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models have shown remarkable success across scientific domains, yet their impact in chemistry remains limited due to the absence of diverse, large-scale, high-quality datasets that reflect the field's multifaceted nature. We present the ChemPile, an open dataset containing over 75 billion tokens of curated chemical data, specifically built for training and evaluating general-purpose models in the chemical sciences. The dataset mirrors the human learning journey through chemistry -- from educational foundations to specialized expertise -- spanning multiple modalities and content types including structured data in diverse chemical representations (SMILES, SELFIES, IUPAC names, InChI, molecular renderings), scientific and educational text, executable code, and chemical images. ChemPile integrates foundational knowledge (textbooks, lecture notes), specialized expertise (scientific articles and language-interfaced data), visual understanding (molecular structures, diagrams), and advanced reasoning (problem-solving traces and code) -- mirroring how human chemists develop expertise through diverse learning materials and experiences. Constructed through hundreds of hours of expert curation, the ChemPile captures both foundational concepts and domain-specific complexity. We provide standardized training, validation, and test splits, enabling robust benchmarking. ChemPile is openly released via HuggingFace with a consistent API, permissive license, and detailed documentation. We hope the ChemPile will serve as a catalyst for chemical AI, enabling the development of the next generation of chemical foundation models.

Related papers

• Benchmarking Multimodal LLMs on Recognition and Understanding over Chemical Tables [48.39080455781475]
  ChemTable is a large-scale benchmark of real-world chemical tables curated from the experimental sections of literature.<n>ChemTable includes expert-annotated cell polygons, logical layouts, and domain-specific labels, including reagents, catalysts, yields, and graphical components.<n>We evaluated a range of representative multimodal models, including both open-source and closed-source models, on ChemTable and reported a series of findings with practical and conceptual insights.
  arXiv  Detail & Related papers  (2025-06-13T00:45:41Z)
• From Generalist to Specialist: A Survey of Large Language Models for Chemistry [14.317448405387195]
  Large Language Models (LLMs) have significantly transformed our daily life and established a new paradigm in natural language processing (NLP)<n>The predominant pretraining of LLMs on extensive web-based texts remains insufficient for advanced scientific discovery, particularly in chemistry.<n>Although several studies have reviewed Pretrained Language Models (PLMs) in chemistry, there is a conspicuous absence of a systematic survey specifically focused on chemistry-oriented LLMs.
  arXiv  Detail & Related papers  (2024-12-28T03:40:25Z)
• MolParser: End-to-end Visual Recognition of Molecule Structures in the Wild [23.558032054114577]
  We present Mol, a novel end-to-end optical chemical structure recognition method.<n>We use a SMILES encoding rule to annotate Mol-7M, the largest annotated molecular image dataset.<n>We trained an end-to-end molecular image captioning model, Mol, using a curriculum learning approach.
  arXiv  Detail & Related papers  (2024-11-17T15:00:09Z)
• ChemVLM: Exploring the Power of Multimodal Large Language Models in Chemistry Area [50.15254966969718]
  We introduce textbfChemVLM, an open-source chemical multimodal large language model for chemical applications.<n>ChemVLM is trained on a carefully curated bilingual dataset that enhances its ability to understand both textual and visual chemical information.<n>We benchmark ChemVLM against a range of open-source and proprietary multimodal large language models on various tasks.
  arXiv  Detail & Related papers  (2024-08-14T01:16:40Z)
• CEAR: Automatic construction of a knowledge graph of chemical entities and roles from scientific literature [4.086092284014203]
  We propose a methodology that involves augmenting existing annotated text corpora with knowledge from Chebi and fine-tuning a large model (LLM) to recognize chemical entities and their roles in scientific text. By combining ontological knowledge understanding capabilities of LLMs, we achieve high precision and recall rates in identifying both the chemical entities and roles in scientific literature.
  arXiv  Detail & Related papers  (2024-07-31T15:56:06Z)
• ChemLLM: A Chemical Large Language Model [49.308528569982805]
  Large language models (LLMs) have made impressive progress in chemistry applications. However, the community lacks an LLM specifically designed for chemistry. Here, we introduce ChemLLM, a comprehensive framework that features the first LLM dedicated to chemistry.
  arXiv  Detail & Related papers  (2024-02-10T01:11:59Z)
• MolGrapher: Graph-based Visual Recognition of Chemical Structures [50.13749978547401]
  We introduce MolGrapher to recognize chemical structures visually. We treat all candidate atoms and bonds as nodes and put them in a graph. We classify atom and bond nodes in the graph with a Graph Neural Network.
  arXiv  Detail & Related papers  (2023-08-23T16:16:11Z)
• Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editing [107.49804059269212]
  We present a multi-modal molecule structure-text model, MoleculeSTM, by jointly learning molecules' chemical structures and textual descriptions. In experiments, MoleculeSTM obtains the state-of-the-art generalization ability to novel biochemical concepts.
  arXiv  Detail & Related papers  (2022-12-21T06:18:31Z)
• A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language [63.60376252491507]
  We propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine.
  arXiv  Detail & Related papers  (2022-09-12T00:56:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.