Recurrent neural network-based volumetric fluorescence microscopy
- URL: http://arxiv.org/abs/2010.10781v1
- Date: Wed, 21 Oct 2020 06:17:38 GMT
- Title: Recurrent neural network-based volumetric fluorescence microscopy
- Authors: Luzhe Huang, Yilin Luo, Yair Rivenson, Aydogan Ozcan
- Abstract summary: We report a deep learning-based image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope.
Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume.
Recurrent-MZ is demonstrated to increase the depth-of-field of a 63xNA objective lens by approximately 50-fold, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume.
- Score: 0.30586855806896046
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Volumetric imaging of samples using fluorescence microscopy plays an
important role in various fields including physical, medical and life sciences.
Here we report a deep learning-based volumetric image inference framework that
uses 2D images that are sparsely captured by a standard wide-field fluorescence
microscope at arbitrary axial positions within the sample volume. Through a
recurrent convolutional neural network, which we term as Recurrent-MZ, 2D
fluorescence information from a few axial planes within the sample is
explicitly incorporated to digitally reconstruct the sample volume over an
extended depth-of-field. Using experiments on C. Elegans and nanobead samples,
Recurrent-MZ is demonstrated to increase the depth-of-field of a 63x/1.4NA
objective lens by approximately 50-fold, also providing a 30-fold reduction in
the number of axial scans required to image the same sample volume. We further
illustrated the generalization of this recurrent network for 3D imaging by
showing its resilience to varying imaging conditions, including e.g., different
sequences of input images, covering various axial permutations and unknown
axial positioning errors. Recurrent-MZ demonstrates the first application of
recurrent neural networks in microscopic image reconstruction and provides a
flexible and rapid volumetric imaging framework, overcoming the limitations of
current 3D scanning microscopy tools.
Related papers
- A Unified Model for Compressed Sensing MRI Across Undersampling Patterns [69.19631302047569]
Deep neural networks have shown great potential for reconstructing high-fidelity images from undersampled measurements.
Our model is based on neural operators, a discretization-agnostic architecture.
Our inference speed is also 1,400x faster than diffusion methods.
arXiv Detail & Related papers (2024-10-05T20:03:57Z) - Style transfer between Microscopy and Magnetic Resonance Imaging via
Generative Adversarial Network in small sample size settings [49.84018914962972]
Cross-modal augmentation of Magnetic Resonance Imaging (MRI) and microscopic imaging based on the same tissue samples is promising.
We tested a method for generating microscopic histological images from MRI scans of the corpus callosum using conditional generative adversarial network (cGAN) architecture.
arXiv Detail & Related papers (2023-10-16T13:58:53Z) - Fast light-field 3D microscopy with out-of-distribution detection and
adaptation through Conditional Normalizing Flows [16.928404625892625]
Real-time 3D fluorescence microscopy is crucial for the analysis of live organisms.
We propose a novel architecture to perform fast 3D reconstructions of live immobilized zebrafish neural activity.
arXiv Detail & Related papers (2023-06-10T10:42:49Z) - Computational 3D topographic microscopy from terabytes of data per
sample [2.4657541547959387]
We present a large-scale computational 3D topographic microscope that enables 6-gigapixel profilometric 3D imaging at micron-scale resolution.
We developed a self-supervised neural network-based algorithm for 3D reconstruction and stitching that jointly estimates an all-in-focus photometric composite and 3D height map.
To demonstrate the broad utility of our new computational microscope, we applied STARCAM to a variety of decimeter-scale objects.
arXiv Detail & Related papers (2023-06-05T07:09:21Z) - Low dosage 3D volume fluorescence microscopy imaging using compressive
sensing [0.0]
We present a compressive sensing (CS) based approach to fully reconstruct 3D volumes with the same signal-to-noise ratio (SNR) with less than half of the excitation dosage.
We demonstrate our technique by capturing a 3D volume of the RFP labeled neurons in the zebrafish embryo spinal cord with the axial sampling of 0.1um using a confocal microscope.
The developed CS-based methodology in this work can be easily applied to other deep imaging modalities such as two-photon and light-sheet microscopy, where reducing sample photo-toxicity is a critical challenge.
arXiv Detail & Related papers (2022-01-03T18:44:50Z) - Deep learning of multi-resolution X-Ray micro-CT images for multi-scale
modelling [0.0]
We develop a 3D Enhanced Deep Super Resolution (EDSR) convolutional neural network to create enhanced, high-resolution data over large spatial scales.
We validate the network with various metrics: textual analysis, segmentation behaviour and pore-network model (PNM) multiphase flow simulations.
The EDSR generated model is more accurate than the base LR model at predicting experimental behaviour in the presence of heterogeneities.
arXiv Detail & Related papers (2021-11-01T21:49:22Z) - 3D Reconstruction of Curvilinear Structures with Stereo Matching
DeepConvolutional Neural Networks [52.710012864395246]
We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs.
We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
arXiv Detail & Related papers (2021-10-14T23:05:47Z) - Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
We present a modern solution to the multi-view photometric stereo problem (MVPS)
We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry.
Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
arXiv Detail & Related papers (2021-10-11T20:20:03Z) - Programmable 3D snapshot microscopy with Fourier convolutional networks [3.2156268397508314]
3D snapshot microscopy enables volumetric imaging as fast as a camera allows by capturing a 3D volume in a single 2D camera image.
We introduce a class of global kernel Fourier convolutional neural networks which can efficiently integrate the globally mixed information encoded in a 3D snapshot image.
arXiv Detail & Related papers (2021-04-21T16:09:56Z) - 4D Spatio-Temporal Convolutional Networks for Object Position Estimation
in OCT Volumes [69.62333053044712]
3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single OCT images.
We extend 3D CNNs to 4D-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking.
arXiv Detail & Related papers (2020-07-02T12:02:20Z) - Multifold Acceleration of Diffusion MRI via Slice-Interleaved Diffusion
Encoding (SIDE) [50.65891535040752]
We propose a diffusion encoding scheme, called Slice-Interleaved Diffusion.
SIDE, that interleaves each diffusion-weighted (DW) image volume with slices encoded with different diffusion gradients.
We also present a method based on deep learning for effective reconstruction of DW images from the highly slice-undersampled data.
arXiv Detail & Related papers (2020-02-25T14:48:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.