Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images

• URL: http://arxiv.org/abs/2301.07743v2
• Date: Mon, 29 May 2023 20:49:31 GMT
• Title: Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images
• Authors: Abid Khan, Chia-Hao Lee, Pinshane Y. Huang, and Bryan K. Clark
• Abstract summary: Machine learning could revolutionize materials research through autonomous data collection and processing. We employ a cycle generative adversarial network (CycleGAN) with a reciprocal space discriminator to augment simulated data with realistic spatial frequency information. We showcase our approach by training a fully convolutional network (FCN) to identify single atom defects in a 4.5 million atom data set.
• Score: 2.5954872177280346
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: The rise of automation and machine learning (ML) in electron microscopy has the potential to revolutionize materials research through autonomous data collection and processing. A significant challenge lies in developing ML models that rapidly generalize to large data sets under varying experimental conditions. We address this by employing a cycle generative adversarial network (CycleGAN) with a reciprocal space discriminator, which augments simulated data with realistic spatial frequency information. This allows the CycleGAN to generate images nearly indistinguishable from real data and provide labels for ML applications. We showcase our approach by training a fully convolutional network (FCN) to identify single atom defects in a 4.5 million atom data set, collected using automated acquisition in an aberration-corrected scanning transmission electron microscope (STEM). Our method produces adaptable FCNs that can adjust to dynamically changing experimental variables with minimal intervention, marking a crucial step towards fully autonomous harnessing of microscopy big data.

Related papers

• Learning and Controlling Silicon Dopant Transitions in Graphene using Scanning Transmission Electron Microscopy [58.51812955462815]
  We introduce a machine learning approach to determine the transition dynamics of silicon atoms on a single layer of carbon atoms. The data samples are processed and filtered to produce symbolic representations, which we use to train a neural network to predict transition probabilities. These learned transition dynamics are then leveraged to guide a single silicon atom throughout the lattice to pre-determined target destinations.
  arXiv  Detail & Related papers  (2023-11-21T21:51:00Z)
• Closing the loop: Autonomous experiments enabled by machine-learning-based online data analysis in synchrotron beamline environments [80.49514665620008]
  Machine learning can be used to enhance research involving large or rapidly generated datasets. In this study, we describe the incorporation of ML into a closed-loop workflow for X-ray reflectometry (XRR) We present solutions that provide an elementary data analysis in real time during the experiment without introducing the additional software dependencies in the beamline control software environment.
  arXiv  Detail & Related papers  (2023-06-20T21:21:19Z)
• Convolutional Monge Mapping Normalization for learning on sleep data [63.22081662149488]
  We propose a new method called Convolutional Monge Mapping Normalization (CMMN) CMMN consists in filtering the signals in order to adapt their power spectrum density (PSD) to a Wasserstein barycenter estimated on training data. Numerical experiments on sleep EEG data show that CMMN leads to significant and consistent performance gains independent from the neural network architecture.
  arXiv  Detail & Related papers  (2023-05-30T08:24:01Z)
• Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy [0.0]
  Machine learning (ML) has become critical for post-acquisition data analysis in () transmission electron microscopy,scanning (S)TEM, imaging and spectroscopy. We discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects. These considerations will collectively inform the operationalization of ML in next-generation experimentation.
  arXiv  Detail & Related papers  (2023-04-04T18:01:56Z)
• Towards Augmented Microscopy with Reinforcement Learning-Enhanced Workflows [0.0]
  We develop a virtual environment to test and develop a network to autonomously align the electron beam without prior knowledge. We deploy a successful model on the microscope to validate the approach and demonstrate the value of designing appropriate virtual environments. Overall, the results highlight that by taking advantage of RL, microscope operations can be automated without the need for extensive algorithm design.
  arXiv  Detail & Related papers  (2022-08-04T20:02:21Z)
• Disentangling multiple scattering with deep learning: application to strain mapping from electron diffraction patterns [48.53244254413104]
  We implement a deep neural network called FCU-Net to invert highly nonlinear electron diffraction patterns into quantitative structure factor images. We trained the FCU-Net using over 200,000 unique dynamical diffraction patterns which include many different combinations of crystal structures. Our simulated diffraction pattern library, implementation of FCU-Net, and trained model weights are freely available in open source repositories.
  arXiv  Detail & Related papers  (2022-02-01T03:53:39Z)
• Learning to automate cryo-electron microscopy data collection with Ptolemy [4.6453787256723365]
  cryogenic electron microscopy (cryo-EM) has emerged as a primary method for determining near-native, near-atomic resolution 3D structures of biological macromolecules. Currently, the process of collecting high-magnification cryo-EM micrographs requires human input and manual tuning of parameters. Here, we develop the first pipeline to automate low- and medium-magnification targeting with purpose-built algorithms.
  arXiv  Detail & Related papers  (2021-12-01T22:39:28Z)
• 3D fluorescence microscopy data synthesis for segmentation and benchmarking [0.9922927990501083]
  Conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms.
  arXiv  Detail & Related papers  (2021-07-21T16:08:56Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• Global Voxel Transformer Networks for Augmented Microscopy [54.730707387866076]
  We introduce global voxel transformer networks (GVTNets), an advanced deep learning tool for augmented microscopy. GVTNets are built on global voxel transformer operators (GVTOs), which are able to aggregate global information. We apply the proposed methods on existing datasets for three different augmented microscopy tasks under various settings.
  arXiv  Detail & Related papers  (2020-08-05T20:11:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.