Foundation-Model Surrogates Enable Data-Efficient Active Learning for Materials Discovery

• URL: http://arxiv.org/abs/2603.12567v2
• Date: Tue, 17 Mar 2026 08:43:26 GMT
• Title: Foundation-Model Surrogates Enable Data-Efficient Active Learning for Materials Discovery
• Abstract summary: In-Context Active Learning replaces conventional surrogates with TabPFN.<n> TabPFN is a transformer-based foundation model pre-trained on millions of synthetic regression tasks.<n>It delivers strong small-data regression performance and well-calibrated predictive uncertainty.
• Score: 12.870479903068665
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Active learning (AL) has emerged as a powerful paradigm for accelerating materials discovery by iteratively steering experiments toward promising candidates, reducing the number of costly synthesis-and-characterization cycles needed to identify optimal materials. However, current AL relies predominantly on Gaussian Process (GP) and Random Forest (RF) surrogates, which suffer from complementary limitations: GP underfits complex composition-property landscapes due to rigid kernel assumptions, while RF produces unreliable heuristic uncertainty estimates in small-data regimes. This small-data challenge is pervasive in materials science, making reliable surrogate modeling extremely difficult with models trained from scratch on each new dataset. Here we propose In-Context Active Learning (ICAL), which addresses this bottleneck by replacing conventional surrogates with TabPFN, a transformer-based foundation model (FM) pre-trained on millions of synthetic regression tasks to meta-learn a universal prior over tabular data, upon which TabPFN performs principled Bayesian inference in a single forward pass without dataset-specific retraining, delivering strong small-data regression performance and well-calibrated predictive uncertainty (required for effective AL). We benchmark ICAL against GP and RF across 10 materials datasets and TabPFN wins on 8 out of 10 datasets, achieving a mean saving of 52% in extra evaluations relative to GP and 29.77% relative to RF. Cross-validation analysis confirms that TabPFN's advantage stems from superior uncertainty calibration, achieving the lowest Negative Log-Likelihood and Area Under the Sparsification Error curve among all surrogates. These results demonstrate that pre-trained FMs can serve as effective surrogates for active learning, enabling data-efficient discovery across diverse materials systems and small-data experimental sciences.

Related papers

• Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols [123.73663884421272]
  Few-shot transfer has been revolutionized by stronger pre-trained models and improved adaptation algorithms.<n>We establish FEWTRANS, a comprehensive benchmark containing 10 diverse datasets.<n>By releasing FEWTRANS, we aim to provide a rigorous "ruler" to streamline reproducible advances in few-shot transfer learning research.
  arXiv  Detail & Related papers  (2026-02-28T05:41:57Z)
• Robust Machine Learning Framework for Reliable Discovery of High-Performance Half-Heusler Thermoelectrics [0.0]
  Machine learning (ML) can facilitate efficient thermoelectric (TE) material discovery essential to address the environmental crisis.<n>ML models often suffer from poor experimental generalizability despite high metrics.<n>This study presents a robust workflow, applied to the half-Heusler (hH) structural prototype, for figure of merit (zT) prediction.
  arXiv  Detail & Related papers  (2026-02-01T10:50:42Z)
• Quantum-Aware Generative AI for Materials Discovery: A Framework for Robust Exploration Beyond DFT Biases [0.0]
  We introduce a quantum-aware generative AI framework for materials discovery.<n>We implement a robust active learning loop that quantifies and targets the divergence between low- and high-fidelity predictions.<n>Our results demonstrate a 3-5x improvement in successfully identifying potentially stable candidates in high-divergence regions.
  arXiv  Detail & Related papers  (2025-12-13T11:17:21Z)
• Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging [40.35825564674249]
  This study introduces the first structured benchmark to assess the robustness and efficiency of transfer learning strategies for Foundation Models.<n>Four publicly available COVID-19 chest X-ray datasets were used, covering mortality, severity, and admission.<n>CNNs pretrained on ImageNet and FMs pretrained on general or biomedical datasets were adapted using full finetuning, linear probing, and parameter-efficient methods.
  arXiv  Detail & Related papers  (2025-06-23T09:16:04Z)
• Model-agnostic Mitigation Strategies of Data Imbalance for Regression [0.0]
  Data imbalance persists as a pervasive challenge in regression tasks, introducing bias in model performance and undermining predictive reliability.<n>We present advanced mitigation techniques, which build upon and improve existing sampling methods.<n>We demonstrate that constructing an ensemble of models -- one trained with imbalance mitigation and another without -- can significantly reduce these negative effects.
  arXiv  Detail & Related papers  (2025-06-02T09:46:08Z)
• Advancing Tabular Stroke Modelling Through a Novel Hybrid Architecture and Feature-Selection Synergy [0.9999629695552196]
  The present work develops and validates a data-driven and interpretable machine-learning framework designed to predict strokes.<n>Ten routinely gathered demographic, lifestyle, and clinical variables were sourced from a public cohort of 4,981 records.<n>The proposed model achieved an accuracy rate of 97.2% and an F1-score of 97.15%, indicating a significant enhancement compared to the leading individual model.
  arXiv  Detail & Related papers  (2025-05-18T21:46:45Z)
• CALF: A Conditionally Adaptive Loss Function to Mitigate Class-Imbalanced Segmentation [0.2902243522110345]
  Imbalanced datasets pose a challenge in training deep learning (DL) models for medical diagnostics.<n>We propose a novel, statistically driven, conditionally adaptive loss function (CALF) tailored to accommodate the conditions of imbalanced datasets in DL training.
  arXiv  Detail & Related papers  (2025-04-06T12:03:33Z)
• DispFormer: A Pretrained Transformer Incorporating Physical Constraints for Dispersion Curve Inversion [56.64622091009756]
  This study introduces DispFormer, a transformer-based neural network for $v_s$ profile inversion from Rayleigh-wave phase and group dispersion curves.<n>DispFormer processes dispersion data independently at each period, allowing it to handle varying lengths without requiring network modifications or strict alignment between training and testing datasets.
  arXiv  Detail & Related papers  (2025-01-08T09:08:24Z)
• Transfer Learning for Deep Learning-based Prediction of Lattice Thermal Conductivity [0.0]
  We study the impact of transfer learning on the precision and generalizability of a deep learning model (ParAIsite)<n>We show that a much greater improvement is obtained when first fine-tuning it on a large datasets of low-quality approximations of lattice thermal conductivity (LTC)<n>The promising results pave the way towards a greater ability to explore large databases in search of low thermal conductivity materials.
  arXiv  Detail & Related papers  (2024-11-27T11:57:58Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Test-Time Adaptation Induces Stronger Accuracy and Agreement-on-the-Line [65.14099135546594]
  Recent test-time adaptation (TTA) methods drastically strengthen the ACL and AGL trends in models, even in shifts where models showed very weak correlations before. Our results show that by combining TTA with AGL-based estimation methods, we can estimate the OOD performance of models with high precision for a broader set of distribution shifts.
  arXiv  Detail & Related papers  (2023-10-07T23:21:25Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Machine-Learning Prediction of the Computed Band Gaps of Double Perovskite Materials [3.2798940914359056]
  Prediction of the electronic structure of functional materials is essential for the engineering of new devices. In this study, we use machine learning to predict the electronic structure of double perovskite materials. Our results are significant in the sense that they attest to the potential of machine learning regressions for the rapid screening of promising candidate functional materials.
  arXiv  Detail & Related papers  (2023-01-04T08:19:18Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Semantic Perturbations with Normalizing Flows for Improved Generalization [62.998818375912506]
  We show that perturbations in the latent space can be used to define fully unsupervised data augmentations. We find that our latent adversarial perturbations adaptive to the classifier throughout its training are most effective.
  arXiv  Detail & Related papers  (2021-08-18T03:20:00Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Deep Learning for Virtual Screening: Five Reasons to Use ROC Cost Functions [80.12620331438052]
  deep learning has become an important tool for rapid screening of billions of molecules in silico for potential hits containing desired chemical features. Despite its importance, substantial challenges persist in training these models, such as severe class imbalance, high decision thresholds, and lack of ground truth labels in some datasets. We argue in favor of directly optimizing the receiver operating characteristic (ROC) in such cases, due to its robustness to class imbalance.
  arXiv  Detail & Related papers  (2020-06-25T08:46:37Z)
• Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees [49.91477656517431]
  Quantization-based solvers have been widely adopted in Federated Learning (FL) No existing methods enjoy all the aforementioned properties. We propose an intuitively-simple yet theoretically-simple method based on SIGNSGD to bridge the gap.
  arXiv  Detail & Related papers  (2020-02-25T15:12:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.