Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks

• URL: http://arxiv.org/abs/2503.03507v1
• Date: Wed, 05 Mar 2025 13:55:26 GMT
• Title: Mineral segmentation using electron microscope images and spectral sampling through multimodal graph neural networks
• Authors: Samuel Repka, Bořek Reich, Fedor Zolotarev, Tuomas Eerola, Pavel Zemčík,
• Abstract summary: We propose a novel Graph Neural Network-based method for segmentation based on data fusion of multimodal Scanning Electron Microscope (SEM) images.<n>Our results demonstrate that providing EDS data for as few as 1% of BSE pixels produces accurate segmentation.
• Score: 0.03994567502796063
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a novel Graph Neural Network-based method for segmentation based on data fusion of multimodal Scanning Electron Microscope (SEM) images. In most cases, Backscattered Electron (BSE) images obtained using SEM do not contain sufficient information for mineral segmentation. Therefore, imaging is often complemented with point-wise Energy-Dispersive X-ray Spectroscopy (EDS) spectral measurements that provide highly accurate information about the chemical composition but that are time-consuming to acquire. This motivates the use of sparse spectral data in conjunction with BSE images for mineral segmentation. The unstructured nature of the spectral data makes most traditional image fusion techniques unsuitable for BSE-EDS fusion. We propose using graph neural networks to fuse the two modalities and segment the mineral phases simultaneously. Our results demonstrate that providing EDS data for as few as 1% of BSE pixels produces accurate segmentation, enabling rapid analysis of mineral samples. The proposed data fusion pipeline is versatile and can be adapted to other domains that involve image data and point-wise measurements.

Related papers

• MRGen: Segmentation Data Engine For Underrepresented MRI Modalities [59.61465292965639]
  Training medical image segmentation models for rare yet clinically significant imaging modalities is challenging due to the scarcity of annotated data. This paper investigates leveraging generative models to synthesize training data, to train segmentation models for underrepresented modalities.
  arXiv  Detail & Related papers  (2024-12-04T16:34:22Z)
• Spectral Image Data Fusion for Multisource Data Augmentation [44.99833362998488]
  Multispectral and hyperspectral images are increasingly popular in different research fields, such as remote sensing, astronomical imaging, or precision agriculture. The amount of free data available to perform machine learning tasks is relatively small. Artificial intelligence models developed in the area of spectral imaging require input images with a fixed spectral signature.
  arXiv  Detail & Related papers  (2024-04-05T13:40:18Z)
• Style transfer between Microscopy and Magnetic Resonance Imaging via Generative Adversarial Network in small sample size settings [49.84018914962972]
  Cross-modal augmentation of Magnetic Resonance Imaging (MRI) and microscopic imaging based on the same tissue samples is promising. We tested a method for generating microscopic histological images from MRI scans of the corpus callosum using conditional generative adversarial network (cGAN) architecture.
  arXiv  Detail & Related papers  (2023-10-16T13:58:53Z)
• Toward deep-learning-assisted spectrally-resolved imaging of magnetic noise [52.77024349608834]
  We implement a deep neural network to efficiently reconstruct the spectral density of the underlying fluctuating magnetic field. These results create opportunities for the application of machine-learning methods to color-center-based nanoscale sensing and imaging.
  arXiv  Detail & Related papers  (2022-08-01T19:18:26Z)
• Classification of FIB/SEM-tomography images for highly porous multiphase materials using random forest classifiers [0.0]
  We present a novel approach for data classification in three-dimensional image data obtained by FIB/SEM tomography. We use two different image signals, namely the signal of the angled SE2 chamber detector and the Inlens detector signal, combine both signals and train a random forest. This approach may yield as guideline for future research using FIB/SEM tomography.
  arXiv  Detail & Related papers  (2022-07-28T14:28:30Z)
• Negligible effect of brain MRI data preprocessing for tumor segmentation [36.89606202543839]
  We conduct experiments on three publicly available datasets and evaluate the effect of different preprocessing steps in deep neural networks. Our results demonstrate that most popular standardization steps add no value to the network performance. We suggest that image intensity normalization approaches do not contribute to model accuracy because of the reduction of signal variance with image standardization.
  arXiv  Detail & Related papers  (2022-04-11T17:29:36Z)
• Hyperspectral Image Segmentation based on Graph Processing over Multilayer Networks [51.15952040322895]
  One important task of hyperspectral image (HSI) processing is the extraction of spectral-spatial features. We propose several approaches to HSI segmentation based on M-GSP feature extraction. Our experimental results demonstrate the strength of M-GSP in HSI processing and spectral-spatial information extraction.
  arXiv  Detail & Related papers  (2021-11-29T23:28:18Z)
• Improving the Segmentation of Scanning Probe Microscope Images using Convolutional Neural Networks [0.9236074230806579]
  We develop protocols for the segmentation of images of 2D assemblies of gold nanoparticles formed on silicon surfaces via deposition from an organic solvent. The evaporation of the solvent drives far-from-equilibrium self-organisation of the particles, producing a wide variety of nano- and micro-structured patterns. We show that a segmentation strategy using the U-Net convolutional neural network outperforms traditional automated approaches.
  arXiv  Detail & Related papers  (2020-08-27T20:49:59Z)
• Data-Driven Discovery of Molecular Photoswitches with Multioutput Gaussian Processes [51.17758371472664]
  Photoswitchable molecules display two or more isomeric forms that may be accessed using light. We present a data-driven discovery pipeline for molecular photoswitches underpinned by dataset curation and multitask learning. We validate our proposed approach experimentally by screening a library of commercially available photoswitchable molecules.
  arXiv  Detail & Related papers  (2020-06-28T20:59:03Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)
• Fast reconstruction of atomic-scale STEM-EELS images from sparse sampling [11.624024992823928]
  We propose a fast and accurate reconstruction method suited for atomic-scale EELS. This method is compared to popular solutions such as beta process factor analysis (BPFA) which is used for the first time on STEM-EELS images.
  arXiv  Detail & Related papers  (2020-02-04T11:07:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.