Human-in-the-loop: The future of Machine Learning in Automated Electron Microscopy

• URL: http://arxiv.org/abs/2310.05018v1
• Date: Sun, 8 Oct 2023 05:26:32 GMT
• Title: Human-in-the-loop: The future of Machine Learning in Automated Electron Microscopy
• Authors: Sergei V. Kalinin, Yongtao Liu, Arpan Biswas, Gerd Duscher, Utkarsh Pratiush, Kevin Roccapriore, Maxim Ziatdinov and Rama Vasudevan
• Abstract summary: We discuss some considerations in designing ML-based active experiments. The likely strategy for the next several years will be human-in-the-loop automated experiments.
• Score: 0.6760163180787716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning methods are progressively gaining acceptance in the electron microscopy community for de-noising, semantic segmentation, and dimensionality reduction of data post-acquisition. The introduction of the APIs by major instrument manufacturers now allows the deployment of ML workflows in microscopes, not only for data analytics but also for real-time decision-making and feedback for microscope operation. However, the number of use cases for real-time ML remains remarkably small. Here, we discuss some considerations in designing ML-based active experiments and pose that the likely strategy for the next several years will be human-in-the-loop automated experiments (hAE). In this paradigm, the ML learning agent directly controls beam position and image and spectroscopy acquisition functions, and human operator monitors experiment progression in real- and feature space of the system and tunes the policies of the ML agent to steer the experiment towards specific objectives.

Related papers

• MLXP: A Framework for Conducting Replicable Experiments in Python [63.37350735954699]
  We propose MLXP, an open-source, simple, and lightweight experiment management tool based on Python. It streamlines the experimental process with minimal overhead while ensuring a high level of practitioner overhead.
  arXiv  Detail & Related papers  (2024-02-21T14:22:20Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• Closing the loop: Autonomous experiments enabled by machine-learning-based online data analysis in synchrotron beamline environments [80.49514665620008]
  Machine learning can be used to enhance research involving large or rapidly generated datasets. In this study, we describe the incorporation of ML into a closed-loop workflow for X-ray reflectometry (XRR) We present solutions that provide an elementary data analysis in real time during the experiment without introducing the additional software dependencies in the beamline control software environment.
  arXiv  Detail & Related papers  (2023-06-20T21:21:19Z)
• Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy [0.0]
  Machine learning (ML) has become critical for post-acquisition data analysis in () transmission electron microscopy,scanning (S)TEM, imaging and spectroscopy. We discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects. These considerations will collectively inform the operationalization of ML in next-generation experimentation.
  arXiv  Detail & Related papers  (2023-04-04T18:01:56Z)
• Microscopy is All You Need [0.0]
  We argue that a promising pathway for the development of machine learning methods is via the route of domain-specific deployable algorithms. This will benefit both fundamental physical studies and serve as a test bed for more complex autonomous systems such as robotics and manufacturing.
  arXiv  Detail & Related papers  (2022-10-12T18:41:40Z)
• Machine learning in bioprocess development: From promise to practice [58.720142291102135]
  Data-driven methods like machine learning (ML) approaches have a high potential to rationally explore large design spaces. The aim of this review is to demonstrate how ML methods have been applied so far in bioprocess development.
  arXiv  Detail & Related papers  (2022-10-04T13:48:59Z)
• Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learning [0.0]
  We discuss the basic principles of Bayesian active learning and illustrate its applications for scanning probe microscopes (SPMs) These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment.
  arXiv  Detail & Related papers  (2022-05-30T23:01:41Z)
• Machine learning enabling high-throughput and remote operations at large-scale user facilities [0.0]
  Machine learning (ML) methods are regularly developed to process and interpret large datasets in real-time with measurements. We demonstrate a variety of archetypal ML models for on-the-fly analysis at multiple beamlines at the National Synchrotron Light Source II (NSLS-II)
  arXiv  Detail & Related papers  (2022-01-09T17:43:03Z)
• MAML is a Noisy Contrastive Learner [72.04430033118426]
  Model-agnostic meta-learning (MAML) is one of the most popular and widely-adopted meta-learning algorithms nowadays. We provide a new perspective to the working mechanism of MAML and discover that: MAML is analogous to a meta-learner using a supervised contrastive objective function. We propose a simple but effective technique, zeroing trick, to alleviate such interference.
  arXiv  Detail & Related papers  (2021-06-29T12:52:26Z)
• Automated and Autonomous Experiment in Electron and Scanning Probe Microscopy [0.0]
  We aim to analyze the major pathways towards automated experiment (AE) in imaging methods with sequential image formation mechanisms. We argue that automated experiments should necessarily be discussed in a broader context of the general domain knowledge that both informs the experiment and is increased as the result of the experiment.
  arXiv  Detail & Related papers  (2021-03-22T20:24:41Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.