Recurrent neural network-based volumetric fluorescence microscopy

• URL: http://arxiv.org/abs/2010.10781v1
• Date: Wed, 21 Oct 2020 06:17:38 GMT
• Title: Recurrent neural network-based volumetric fluorescence microscopy
• Authors: Luzhe Huang, Yilin Luo, Yair Rivenson, Aydogan Ozcan
• Abstract summary: We report a deep learning-based image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume. Recurrent-MZ is demonstrated to increase the depth-of-field of a 63xNA objective lens by approximately 50-fold, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume.
• Score: 0.30586855806896046
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over an extended depth-of-field. Using experiments on C. Elegans and nanobead samples, Recurrent-MZ is demonstrated to increase the depth-of-field of a 63x/1.4NA objective lens by approximately 50-fold, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume. We further illustrated the generalization of this recurrent network for 3D imaging by showing its resilience to varying imaging conditions, including e.g., different sequences of input images, covering various axial permutations and unknown axial positioning errors. Recurrent-MZ demonstrates the first application of recurrent neural networks in microscopic image reconstruction and provides a flexible and rapid volumetric imaging framework, overcoming the limitations of current 3D scanning microscopy tools.

Related papers

• Unifying Subsampling Pattern Variations for Compressed Sensing MRI with Neural Operators [72.79532467687427]
  Compressed Sensing MRI reconstructs images of the body's internal anatomy from undersampled and compressed measurements. Deep neural networks have shown great potential for reconstructing high-quality images from highly undersampled measurements. We propose a unified model that is robust to different subsampling patterns and image resolutions in CS-MRI.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• Style transfer between Microscopy and Magnetic Resonance Imaging via Generative Adversarial Network in small sample size settings [49.84018914962972]
  Cross-modal augmentation of Magnetic Resonance Imaging (MRI) and microscopic imaging based on the same tissue samples is promising. We tested a method for generating microscopic histological images from MRI scans of the corpus callosum using conditional generative adversarial network (cGAN) architecture.
  arXiv  Detail & Related papers  (2023-10-16T13:58:53Z)
• Fast light-field 3D microscopy with out-of-distribution detection and adaptation through Conditional Normalizing Flows [16.928404625892625]
  Real-time 3D fluorescence microscopy is crucial for the analysis of live organisms. We propose a novel architecture to perform fast 3D reconstructions of live immobilized zebrafish neural activity.
  arXiv  Detail & Related papers  (2023-06-10T10:42:49Z)
• Computational 3D topographic microscopy from terabytes of data per sample [2.4657541547959387]
  We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution. We developed a self-supervised neural network-based algorithm for 3D reconstruction and stitching that jointly estimates an all-in-focus photometric composite and 3D height map. To demonstrate the broad utility of our new computational microscope, we applied STARCAM to a variety of decimeter-scale objects.
  arXiv  Detail & Related papers  (2023-06-05T07:09:21Z)
• Low dosage 3D volume fluorescence microscopy imaging using compressive sensing [0.0]
  We present a compressive sensing (CS) based approach to fully reconstruct 3D volumes with the same signal-to-noise ratio (SNR) with less than half of the excitation dosage. We demonstrate our technique by capturing a 3D volume of the RFP labeled neurons in the zebrafish embryo spinal cord with the axial sampling of 0.1um using a confocal microscope. The developed CS-based methodology in this work can be easily applied to other deep imaging modalities such as two-photon and light-sheet microscopy, where reducing sample photo-toxicity is a critical challenge.
  arXiv  Detail & Related papers  (2022-01-03T18:44:50Z)
• Deep learning of multi-resolution X-Ray micro-CT images for multi-scale modelling [0.0]
  We develop a 3D Enhanced Deep Super Resolution (EDSR) convolutional neural network to create enhanced, high-resolution data over large spatial scales. We validate the network with various metrics: textual analysis, segmentation behaviour and pore-network model (PNM) multiphase flow simulations. The EDSR generated model is more accurate than the base LR model at predicting experimental behaviour in the presence of heterogeneities.
  arXiv  Detail & Related papers  (2021-11-01T21:49:22Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
  We present a modern solution to the multi-view photometric stereo problem (MVPS) We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry. Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
  arXiv  Detail & Related papers  (2021-10-11T20:20:03Z)
• Programmable 3D snapshot microscopy with Fourier convolutional networks [3.2156268397508314]
  3D snapshot microscopy enables volumetric imaging as fast as a camera allows by capturing a 3D volume in a single 2D camera image. We introduce a class of global kernel Fourier convolutional neural networks which can efficiently integrate the globally mixed information encoded in a 3D snapshot image.
  arXiv  Detail & Related papers  (2021-04-21T16:09:56Z)
• 4D Spatio-Temporal Convolutional Networks for Object Position Estimation in OCT Volumes [69.62333053044712]
  3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single OCT images. We extend 3D CNNs to 4D-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking.
  arXiv  Detail & Related papers  (2020-07-02T12:02:20Z)
• Multifold Acceleration of Diffusion MRI via Slice-Interleaved Diffusion Encoding (SIDE) [50.65891535040752]
  We propose a diffusion encoding scheme, called Slice-Interleaved Diffusion. SIDE, that interleaves each diffusion-weighted (DW) image volume with slices encoded with different diffusion gradients. We also present a method based on deep learning for effective reconstruction of DW images from the highly slice-undersampled data.
  arXiv  Detail & Related papers  (2020-02-25T14:48:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.