A Universal Deep Learning Framework for Materials X-ray Absorption Spectra
- URL: http://arxiv.org/abs/2409.19552v2
- Date: Wed, 13 Nov 2024 17:12:34 GMT
- Title: A Universal Deep Learning Framework for Materials X-ray Absorption Spectra
- Authors: Shubha R. Kharel, Fanchen Meng, Xiaohui Qu, Matthew R. Carbone, Deyu Lu,
- Abstract summary: X-ray absorption spectroscopy (XAS) is a powerful characterization technique for probing the local chemical environment of absorbing atoms.
We present a framework that contains a suite of transfer learning approaches for XAS prediction, each contributing to improved accuracy and efficiency.
Our approach boosts the throughput of XAS modeling by orders of magnitude versus first-principles simulations and is extendable to XAS prediction for a broader range of elements.
- Score: 0.6291443816903801
- License:
- Abstract: X-ray absorption spectroscopy (XAS) is a powerful characterization technique for probing the local chemical environment of absorbing atoms. However, analyzing XAS data presents significant challenges, often requiring extensive, computationally intensive simulations, as well as significant domain expertise. These limitations hinder the development of fast, robust XAS analysis pipelines that are essential in high-throughput studies and for autonomous experimentation. We address these challenges with OmniXAS, a framework that contains a suite of transfer learning approaches for XAS prediction, each contributing to improved accuracy and efficiency, as demonstrated on K-edge spectra database covering eight 3d transition metals (Ti-Cu). The OmniXAS framework is built upon three distinct strategies. First, we use M3GNet to derive latent representations of the local chemical environment of absorption sites as input for XAS prediction, achieving up to order-of-magnitude improvements over conventional featurization techniques. Second, we employ a hierarchical transfer learning strategy, training a universal multi-task model across elements before fine-tuning for element-specific predictions. Models based on this cascaded approach after element-wise fine-tuning outperform element-specific models by up to 69%. Third, we implement cross-fidelity transfer learning, adapting a universal model to predict spectra generated by simulation of a different fidelity with a higher computational cost. This approach improves prediction accuracy by up to 11% over models trained on the target fidelity alone. Our approach boosts the throughput of XAS modeling by orders of magnitude versus first-principles simulations and is extendable to XAS prediction for a broader range of elements. This transfer learning framework is generalizable to enhance deep-learning models that target other properties in materials research.
Related papers
- MAX: Masked Autoencoder for X-ray Fluorescence in Geological Investigation [7.777211995715721]
We propose a scalable self-supervised learner, masked autoencoders on XRF spectra (MAX) to pre-train a foundation model.
We find that masking a high proportion of the input spectrum (50%) yields a nontrivial and meaningful self-supervisory task.
Our results show that MAX, requiring only one-third of the data, outperforms models without pre-training in terms of quantification accuracy.
arXiv Detail & Related papers (2024-10-16T07:52:26Z) - OPUS: Occupancy Prediction Using a Sparse Set [64.60854562502523]
We present a framework to simultaneously predict occupied locations and classes using a set of learnable queries.
OPUS incorporates a suite of non-trivial strategies to enhance model performance.
Our lightest model achieves superior RayIoU on the Occ3D-nuScenes dataset at near 2x FPS, while our heaviest model surpasses previous best results by 6.1 RayIoU.
arXiv Detail & Related papers (2024-09-14T07:44:22Z) - Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF [82.7679132059169]
Reinforcement learning from human feedback has emerged as a central tool for language model alignment.
We propose a new algorithm for online exploration in RLHF, Exploratory Preference Optimization (XPO)
XPO enjoys the strongest known provable guarantees and promising empirical performance.
arXiv Detail & Related papers (2024-05-31T17:39:06Z) - Skeleton2vec: A Self-supervised Learning Framework with Contextualized
Target Representations for Skeleton Sequence [56.092059713922744]
We show that using high-level contextualized features as prediction targets can achieve superior performance.
Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework.
Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
arXiv Detail & Related papers (2024-01-01T12:08:35Z) - Embedded feature selection in LSTM networks with multi-objective
evolutionary ensemble learning for time series forecasting [49.1574468325115]
We present a novel feature selection method embedded in Long Short-Term Memory networks.
Our approach optimize the weights and biases of the LSTM in a partitioned manner.
Experimental evaluations on air quality time series data from Italy and southeast Spain demonstrate that our method substantially improves the ability generalization of conventional LSTMs.
arXiv Detail & Related papers (2023-12-29T08:42:10Z) - An Integrative Paradigm for Enhanced Stroke Prediction: Synergizing
XGBoost and xDeepFM Algorithms [1.064427783926208]
We propose an ensemble model that combines the power of XGBoost and xDeepFM algorithms.
Our work aims to improve upon existing stroke prediction models by achieving higher accuracy and robustness.
arXiv Detail & Related papers (2023-10-25T07:55:02Z) - Towards quantitative precision for ECG analysis: Leveraging state space
models, self-supervision and patient metadata [2.0777058026628583]
We investigate three elements aimed at improving the quantitative accuracy of automatic ECG analysis systems.
First, we exploit structured state space models (SSMs) to capture long-term dependencies in time series data.
Secondly, we demonstrate that self-supervised learning using contrastive predictive coding can further improve the performance of SSMs.
Finally, we incorporate basic demographic metadata alongside the ECG signal as input.
arXiv Detail & Related papers (2023-08-29T13:25:26Z) - Exploring Supervised Machine Learning for Multi-Phase Identification and
Quantification from Powder X-Ray Diffraction Spectra [1.0660480034605242]
Powder X-ray diffraction analysis is a critical component of materials characterization methodologies.
Deep learning has become a prime focus for predicting crystallographic parameters and features from X-ray spectra.
Here, we are interested in conventional supervised learning algorithms in lieu of deep learning for multi-label crystalline phase identification.
arXiv Detail & Related papers (2022-11-16T00:36:13Z) - Pre-training via Denoising for Molecular Property Prediction [53.409242538744444]
We describe a pre-training technique that utilizes large datasets of 3D molecular structures at equilibrium.
Inspired by recent advances in noise regularization, our pre-training objective is based on denoising.
arXiv Detail & Related papers (2022-05-31T22:28:34Z) - Explainable Predictive Modeling for Limited Spectral Data [0.0]
We introduce applying recent explainable AI techniques to interpret the prediction outcomes of high-dimensional and limited spectral data.
Due to instrument resolution limitations, pinpointing important regions of the spectroscopy data creates a pathway to optimize the data collection process.
We specifically design three different scenarios to ensure that the evaluation of ML models is robust for the real-time practice.
arXiv Detail & Related papers (2022-02-09T15:46:17Z) - A multi-stage machine learning model on diagnosis of esophageal
manometry [50.591267188664666]
The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage.
This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
arXiv Detail & Related papers (2021-06-25T20:09:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.