Foundation Models for Discovery and Exploration in Chemical Space

• URL: http://arxiv.org/abs/2510.18900v1
• Date: Mon, 20 Oct 2025 17:56:01 GMT
• Title: Foundation Models for Discovery and Exploration in Chemical Space
• Authors: Alexius Wadell, Anoushka Bhutani, Victor Azumah, Austin R. Ellis-Mohr, Celia Kelly, Hancheng Zhao, Anuj K. Nayak, Kareem Hegazy, Alexander Brace, Hongyi Lin, Murali Emani, Venkatram Vishwanath, Kevin Gering, Melisa Alkan, Tom Gibbs, Jack Wells, Lav R. Varshney, Bharath Ramsundar, Karthik Duraisamy, Michael W. Mahoney, Arvind Ramanathan, Venkatasubramanian Viswanathan,
• Abstract summary: MIST is a family of molecular foundation models trained on large unlabeled datasets.<n>We demonstrate the ability of these models to solve real-world problems across chemical space.
• Score: 57.97784111110166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate prediction of atomistic, thermodynamic, and kinetic properties from molecular structures underpins materials innovation. Existing computational and experimental approaches lack the scalability required to efficiently navigate chemical space. Scientific foundation models trained on large unlabeled datasets offer a path toward exploring chemical space across diverse application domains. Here we develop MIST, a family of molecular foundation models with up to an order of magnitude more parameters and data than prior works. Trained using a novel tokenization scheme that comprehensively captures nuclear, electronic, and geometric information, MIST learns from a diverse range of molecules. MIST models have been fine-tuned to predict more than 400 structure -- property relationships and match or exceed state-of-the-art performance across benchmarks spanning physiology, electrochemistry, and quantum chemistry. We demonstrate the ability of these models to solve real-world problems across chemical space, including multiobjective electrolyte solvent screening, olfactory perception mapping, isotope half-life prediction, stereochemical reasoning for chiral organometallic compounds, and binary and multi-component mixture property prediction. Probing MIST models using mechanistic interpretability methods reveals identifiable patterns and trends not explicitly present in the training data, suggesting that the models learn generalizable scientific concepts. We formulate hyperparameter-penalized Bayesian neural scaling laws and use them to reduce the computational cost of model development by an order of magnitude. The methods and findings presented here represent a significant step toward accelerating materials discovery, design, and optimization using foundation models and provide valuable guidance for training compute-optimal scientific foundation models.

Related papers

• Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis [51.83339196548892]
  ChemCraft is a novel framework that decouples chemical reasoning from knowledge storage.<n>ChemCraft achieves superior performance with minimal inference costs.<n>This work establishes a cost-effective and privacy-preserving paradigm for AI-aided chemistry.
  arXiv  Detail & Related papers  (2026-01-25T04:23:34Z)
• Pre-trained Molecular Language Models with Random Functional Group Masking [54.900360309677794]
  We propose a SMILES-based underlineem Molecular underlineem Language underlineem Model, which randomly masking SMILES subsequences corresponding to specific molecular atoms. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities.
  arXiv  Detail & Related papers  (2024-11-03T01:56:15Z)
• Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction [53.06671763877109]
  We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
  arXiv  Detail & Related papers  (2023-02-04T01:32:40Z)
• Generative structured normalizing flow Gaussian processes applied to spectroscopic data [4.0773490083614075]
  In the physical sciences, limited training data may not adequately characterize future observed data. It is critical that models adequately indicate uncertainty, particularly when they may be asked to extrapolate. We demonstrate the methodology on laser-induced breakdown spectroscopy data from the ChemCam instrument onboard the Mars rover Curiosity.
  arXiv  Detail & Related papers  (2022-12-14T23:57:46Z)
• Calibration and generalizability of probabilistic models on low-data chemical datasets with DIONYSUS [0.0]
  We perform an extensive study of the calibration and generalizability of probabilistic machine learning models on small chemical datasets. We analyse the quality of their predictions and uncertainties in a variety of tasks (binary, regression) and datasets. We offer practical insights into model and feature choice for modelling small chemical datasets, a common scenario in new chemical experiments.
  arXiv  Detail & Related papers  (2022-12-03T08:19:06Z)
• Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model [44.60174246341653]
  We present a novel multimodal molecular pre-trained model that incorporates the modalities of structure and biochemical properties. Our proposed model pipeline of data handling and training objectives aligns the structure/property features in a common embedding space. These contributions emerge synergistic knowledge, allowing us to tackle both multimodal and unimodal downstream tasks through a single model.
  arXiv  Detail & Related papers  (2022-11-19T05:16:08Z)
• Improving Molecular Representation Learning with Metric Learning-enhanced Optimal Transport [49.237577649802034]
  We develop a novel optimal transport-based algorithm termed MROT to enhance their generalization capability for molecular regression problems. MROT significantly outperforms state-of-the-art models, showing promising potential in accelerating the discovery of new substances.
  arXiv  Detail & Related papers  (2022-02-13T04:56:18Z)
• Learning Neural Generative Dynamics for Molecular Conformation Generation [89.03173504444415]
  We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph. We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
  arXiv  Detail & Related papers  (2021-02-20T03:17:58Z)
• Multi-task learning for electronic structure to predict and explore molecular potential energy surfaces [39.228041052681526]
  We refine the OrbNet model to accurately predict energy, forces, and other response properties for molecules. The model is end-to-end differentiable due to the derivation of analytic gradients for all electronic structure terms. It is shown to be transferable across chemical space due to the use of domain-specific features.
  arXiv  Detail & Related papers  (2020-11-05T06:48:46Z)
• Physics-Constrained Predictive Molecular Latent Space Discovery with Graph Scattering Variational Autoencoder [0.0]
  We develop a molecular generative model based on variational inference and graph theory in the small data regime. The model's performance is evaluated by generating molecules with desired target properties.
  arXiv  Detail & Related papers  (2020-09-29T09:05:27Z)
• Graph Neural Network for Hamiltonian-Based Material Property Prediction [56.94118357003096]
  We present and compare several different graph convolution networks that are able to predict the band gap for inorganic materials. The models are developed to incorporate two different features: the information of each orbital itself and the interaction between each other. The results show that our model can get a promising prediction accuracy with cross-validation.
  arXiv  Detail & Related papers  (2020-05-27T13:32:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.