Learning Guided Electron Microscopy with Active Acquisition

• URL: http://arxiv.org/abs/2101.02746v1
• Date: Thu, 7 Jan 2021 20:03:16 GMT
• Title: Learning Guided Electron Microscopy with Active Acquisition
• Authors: Lu Mi, Hao Wang, Yaron Meirovitch, Richard Schalek, Srinivas C. Turaga, Jeff W. Lichtman, Aravinthan D.T. Samuel, Nir Shavit
• Abstract summary: We show how to use deep learning to accelerate and optimize single-beam SEM acquisition of images. Our algorithm rapidly collects an information-lossy image and then applies a novel learning method to identify a small subset of pixels to be collected at higher resolution. We demonstrate the efficacy of this novel technique for active acquisition by speeding up the task of collecting connectomic datasets for neurobiology by up to an order of magnitude.
• Score: 8.181540928891913
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single-beam scanning electron microscopes (SEM) are widely used to acquire massive data sets for biomedical study, material analysis, and fabrication inspection. Datasets are typically acquired with uniform acquisition: applying the electron beam with the same power and duration to all image pixels, even if there is great variety in the pixels' importance for eventual use. Many SEMs are now able to move the beam to any pixel in the field of view without delay, enabling them, in principle, to invest their time budget more effectively with non-uniform imaging. In this paper, we show how to use deep learning to accelerate and optimize single-beam SEM acquisition of images. Our algorithm rapidly collects an information-lossy image (e.g. low resolution) and then applies a novel learning method to identify a small subset of pixels to be collected at higher resolution based on a trade-off between the saliency and spatial diversity. We demonstrate the efficacy of this novel technique for active acquisition by speeding up the task of collecting connectomic datasets for neurobiology by up to an order of magnitude.

Related papers

• The Berkeley Single Cell Computational Microscopy (BSCCM) Dataset [1.53744306569115]
  We introduce the Berkeley Single Cell Computational Microscopy dataset. This dataset contains over 12,000,000 images of 400,000 of individual white blood cells.
  arXiv  Detail & Related papers  (2024-02-09T05:10:53Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• AMIGO: Sparse Multi-Modal Graph Transformer with Shared-Context Processing for Representation Learning of Giga-pixel Images [53.29794593104923]
  We present a novel concept of shared-context processing for whole slide histopathology images. AMIGO uses the celluar graph within the tissue to provide a single representation for a patient. We show that our model is strongly robust to missing information to an extent that it can achieve the same performance with as low as 20% of the data.
  arXiv  Detail & Related papers  (2023-03-01T23:37:45Z)
• SEMPAI: a Self-Enhancing Multi-Photon Artificial Intelligence for prior-informed assessment of muscle function and pathology [48.54269377408277]
  We introduce the Self-Enhancing Multi-Photon Artificial Intelligence (SEMPAI), that integrates hypothesis-driven priors in a data-driven Deep Learning approach. SEMPAI performs joint learning of several tasks to enable prediction for small datasets. SEMPAI outperforms state-of-the-art biomarkers in six of seven predictive tasks, including those with scarce data.
  arXiv  Detail & Related papers  (2022-10-28T17:03:04Z)
• Deep learning at the edge enables real-time streaming ptychographic imaging [7.4083593332068975]
  Coherent microscopy techniques like ptychography are poised to revolutionize nanoscale materials characterization. Traditional approaches no longer suffice for recovering sample images in real-time from high-speed coherent imaging experiments. Here, we demonstrate a workflow that leverages artificial intelligence at the edge and high-performance computing to enable real-time inversion on X-ray ptychography data streamed directly from a detector at up to 2 kHz.
  arXiv  Detail & Related papers  (2022-09-20T02:02:37Z)
• Multi-modal Aggregation Network for Fast MR Imaging [85.25000133194762]
  We propose a novel Multi-modal Aggregation Network, named MANet, which is capable of discovering complementary representations from a fully sampled auxiliary modality. In our MANet, the representations from the fully sampled auxiliary and undersampled target modalities are learned independently through a specific network. Our MANet follows a hybrid domain learning framework, which allows it to simultaneously recover the frequency signal in the $k$-space domain.
  arXiv  Detail & Related papers  (2021-10-15T13:16:59Z)
• Ultrafast Focus Detection for Automated Microscopy [0.0]
  We present a fast out-of-focus detection algorithm for electron microscopy images collected serially. Our technique, Multi-scale Histologic Feature Detection, adapts classical computer vision techniques and is based on detecting various fine-grained histologic features. Tests are performed that demonstrate near-real-time detection of out-of-focus conditions.
  arXiv  Detail & Related papers  (2021-08-26T22:24:41Z)
• A parameter refinement method for Ptychography based on Deep Learning concepts [55.41644538483948]
  coarse parametrisation in propagation distance, position errors and partial coherence frequently menaces the experiment viability. A modern Deep Learning framework is used to correct autonomously the setup incoherences, thus improving the quality of a ptychography reconstruction. We tested our system on both synthetic datasets and also on real data acquired at the TwinMic beamline of the Elettra synchrotron facility.
  arXiv  Detail & Related papers  (2021-05-18T10:15:17Z)
• Deep learning-based super-resolution fluorescence microscopy on small datasets [20.349746411933495]
  Deep learning has shown the potentials to reduce the technical barrier and obtain super-resolution from diffraction-limited images. We demonstrate a new convolutional neural network-based approach that is successfully trained with small datasets and super-resolution images. This model can be applied to other biomedical imaging modalities such as MRI and X-ray imaging, where obtaining large training datasets is challenging.
  arXiv  Detail & Related papers  (2021-03-07T03:17:47Z)
• AS-Net: Fast Photoacoustic Reconstruction with Multi-feature Fusion from Sparse Data [1.7237160821929758]
  Photoacoustic imaging is capable of acquiring high contrast images of optical absorption at depths much greater than traditional optical imaging techniques. In this paper, we employ a novel signal processing method to make sparse PA raw data more suitable for the neural network. We then propose Attention Steered Network (AS-Net) for PA reconstruction with multi-feature fusion.
  arXiv  Detail & Related papers  (2021-01-22T03:49:30Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.