MAX: Masked Autoencoder for X-ray Fluorescence in Geological Investigation

• URL: http://arxiv.org/abs/2410.12330v1
• Date: Wed, 16 Oct 2024 07:52:26 GMT
• Title: MAX: Masked Autoencoder for X-ray Fluorescence in Geological Investigation
• Authors: An-Sheng Lee, Yu-Wen Pao, Hsuan-Tien Lin, Sofia Ya Hsuan Liou,
• Abstract summary: We propose a scalable self-supervised learner, masked autoencoders on XRF spectra (MAX) to pre-train a foundation model. We find that masking a high proportion of the input spectrum (50%) yields a nontrivial and meaningful self-supervisory task. Our results show that MAX, requiring only one-third of the data, outperforms models without pre-training in terms of quantification accuracy.
• Score: 7.777211995715721
• License:
• Abstract: Pre-training foundation models has become the de-facto procedure for deep learning approaches, yet its application remains limited in the geological studies, where in needs of the model transferability to break the shackle of data scarcity. Here we target on the X-ray fluorescence (XRF) scanning data, a standard high-resolution measurement in extensive scientific drilling projects. We propose a scalable self-supervised learner, masked autoencoders on XRF spectra (MAX), to pre-train a foundation model covering geological records from multiple regions of the Pacific and Southern Ocean. In pre-training, we find that masking a high proportion of the input spectrum (50\%) yields a nontrivial and meaningful self-supervisory task. For downstream tasks, we select the quantification of XRF spectra into two costly geochemical measurements, CaCO$_3$ and total organic carbon, due to their importance in understanding the paleo-oceanic carbon system. Our results show that MAX, requiring only one-third of the data, outperforms models without pre-training in terms of quantification accuracy. Additionally, the model's generalizability improves by more than 60\% in zero-shot tests on new materials, with explainability further ensuring its robustness. Thus, our approach offers a promising pathway to overcome data scarcity in geological discovery by leveraging the self-supervised foundation model and fast-acquired XRF scanning data.

Related papers

• MA^2: A Self-Supervised and Motion Augmenting Autoencoder for Gait-Based Automatic Disease Detection [7.483446634501235]
  Ground reaction force (GRF) is the force exerted by the ground on a body in contact with it. GRF-based automatic disease detection (ADD) has become an emerging medical diagnosis method.
  arXiv  Detail & Related papers  (2024-11-05T14:21:01Z)
• Diff-CXR: Report-to-CXR generation through a disease-knowledge enhanced diffusion model [4.507437953126754]
  We propose a disease-knowledge enhanced Diffusion-based TTI learning framework, named Diff-CXR, for medical report-to-CXR generation. Experimentally, our Diff-CXR outperforms previous SOTA medical TTI methods by 33.4% / 8.0% and 23.8% / 56.4% in the FID and mAUC score on MIMIC-CXR and IU-Xray.
  arXiv  Detail & Related papers  (2024-10-26T12:38:12Z)
• A Universal Deep Learning Framework for Materials X-ray Absorption Spectra [0.6291443816903801]
  X-ray absorption spectroscopy (XAS) is a powerful characterization technique for probing the local chemical environment of absorbing atoms. We present a framework that contains a suite of transfer learning approaches for XAS prediction, each contributing to improved accuracy and efficiency. Our approach boosts the throughput of XAS modeling by orders of magnitude versus first-principles simulations and is extendable to XAS prediction for a broader range of elements.
  arXiv  Detail & Related papers  (2024-09-29T04:41:10Z)
• Deep Learning for Gamma-Ray Bursts: A data driven event framework for X/Gamma-Ray analysis in space telescopes [2.4666310814233703]
  This thesis is dedicated to providing an overview of Gamma Ray-Bursts (GRBs), their properties, the instrumentation used to detect them, and Artificial Intelligence (AI) applications. Considering both the current and the next generation of high X-ray monitors, such as Fermi-GBM and HERMES Pathfinder, the research question revolves around the detection of long and faint high-energy transients. To address this, two chapters introduce a new data-driven framework, DeepGRB.
  arXiv  Detail & Related papers  (2024-01-28T11:49:57Z)
• Pre-training via Denoising for Molecular Property Prediction [53.409242538744444]
  We describe a pre-training technique that utilizes large datasets of 3D molecular structures at equilibrium. Inspired by recent advances in noise regularization, our pre-training objective is based on denoising.
  arXiv  Detail & Related papers  (2022-05-31T22:28:34Z)
• Tracking perovskite crystallization via deep learning-based feature detection on 2D X-ray scattering data [137.47124933818066]
  We propose an automated pipeline for the analysis of X-ray diffraction images based on the Faster R-CNN deep learning architecture. We demonstrate our method on real-time tracking of organic-inorganic perovskite structure crystallization and test it on two applications.
  arXiv  Detail & Related papers  (2022-02-22T15:39:00Z)
• Improving Classification Model Performance on Chest X-Rays through Lung Segmentation [63.45024974079371]
  We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
  arXiv  Detail & Related papers  (2022-02-22T15:24:06Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• Identification of complex mixtures for Raman spectroscopy using a novel scheme based on a new multi-label deep neural network [0.0]
  We propose a new scheme based on a constant wavelet transform (CWT) and a deep network for classifying complex mixture. A multi-label deep neural network model (MDNN) is then applied for classifying material. The average detection time obtained from our model is 5.31 s, which is much faster than the detection time of the previously proposed models.
  arXiv  Detail & Related papers  (2020-10-29T14:58:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.