ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images

• URL: http://arxiv.org/abs/2003.11064v1
• Date: Tue, 24 Mar 2020 18:42:23 GMT
• Title: ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images
• Authors: Charles N. Christensen, Edward N. Ward, Pietro Lio, Clemens F. Kaminski
• Abstract summary: Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. ML-SIM is thus robust to noise and irregularities in the illumination patterns of the raw SIM input frames.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging because it allows a doubling of image resolution at speeds compatible for live-cell imaging. However, the reconstruction of SIM images is often slow and prone to artefacts. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. The model is an end-to-end deep residual neural network that is trained on a simulated data set to be free of common SIM artefacts. ML-SIM is thus robust to noise and irregularities in the illumination patterns of the raw SIM input frames. The reconstruction method is widely applicable and does not require the acquisition of experimental training data. Since the training data are generated from simulations of the SIM process on images from generic libraries the method can be efficiently adapted to specific experimental SIM implementations. The reconstruction quality enabled by our method is compared with traditional SIM reconstruction methods, and we demonstrate advantages in terms of noise, reconstruction fidelity and contrast for both simulated and experimental inputs. In addition, reconstruction of one SIM frame typically only takes ~100ms to perform on PCs with modern Nvidia graphics cards, making the technique compatible with real-time imaging. The full implementation and the trained networks are available at http://ML-SIM.com.

Related papers

• SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality [136.82569085134554]
  Foundation models like ChatGPT and Sora that are trained on a huge scale of data have made a revolutionary social impact. It is extremely challenging for sensors in many different fields to collect similar scales of natural images to train strong foundation models. This work presents a simple and effective framework SimMAT to study an open problem: the transferability from vision foundation models trained on natural RGB images to other image modalities of different physical properties.
  arXiv  Detail & Related papers  (2024-09-12T14:38:21Z)
• Bridging the Sim-to-Real Gap with Bayesian Inference [53.61496586090384]
  We present SIM-FSVGD for learning robot dynamics from data. We use low-fidelity physical priors to regularize the training of neural network models. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.
  arXiv  Detail & Related papers  (2024-03-25T11:29:32Z)
• sim2real: Cardiac MR Image Simulation-to-Real Translation via Unsupervised GANs [0.4433315630787158]
  We provide image simulation on virtual XCAT subjects with varying anatomies. We propose sim2real translation network to improve image realism. Our usability experiments suggest that sim2real data exhibits a good potential to augment training data and boost the performance of a segmentation algorithm.
  arXiv  Detail & Related papers  (2022-08-09T16:06:06Z)
• Untrained, physics-informed neural networks for structured illumination microscopy [0.456877715768796]
  We show that we can combine a deep neural network with the forward model of the structured illumination process to reconstruct sub-diffraction images without training data. The resulting physics-informed neural network (PINN) can be optimized on a single set of diffraction limited sub-images.
  arXiv  Detail & Related papers  (2022-07-15T19:02:07Z)
• Dynamic Structured Illumination Microscopy with a Neural Space-time Model [5.048742886625779]
  We propose a new method, Speckle Flow SIM, that models sample motion during the data capture in order to reconstruct dynamic scenes with super-resolution. We demonstrated that Speckle Flow SIM can reconstruct a dynamic scene with deformable motion and 1.88x the-temporal resolution in experiment.
  arXiv  Detail & Related papers  (2022-06-03T05:24:06Z)
• Generic Lithography Modeling with Dual-band Optics-Inspired Neural Networks [52.200624127512874]
  We introduce a dual-band optics-inspired neural network design that considers the optical physics underlying lithography. Our approach yields the first published via/metal layer contour simulation at 1nm2/pixel resolution with any tile size. We also achieve 85X simulation speedup over traditional lithography simulator with 1% accuracy loss.
  arXiv  Detail & Related papers  (2022-03-12T08:08:50Z)
• Spatio-temporal Vision Transformer for Super-resolution Microscopy [2.8348950186890467]
  Structured illumination microscopy (SIM) is an optical super-resolution technique that enables live-cell imaging beyond the diffraction limit. We propose a new transformer-based reconstruction method, VSR-SIM, that uses shifted 3-dimensional window multi-head attention. We demonstrate a use case enabled by VSR-SIM referred to as rolling SIM imaging, which increases temporal resolution in SIM by a factor of 9.
  arXiv  Detail & Related papers  (2022-02-28T19:01:10Z)
• DSSIM: a structural similarity index for floating-point data [68.8204255655161]
  We propose an alternative to the popular SSIM that can be applied directly to the floating point data, which we refer to as the Data SSIM (DSSIM) While we demonstrate the usefulness of the DSSIM in the context of evaluating differences due to lossy compression on large volumes of simulation data, the DSSIM may prove useful for many other applications involving simulation or image data.
  arXiv  Detail & Related papers  (2022-02-05T19:18:33Z)
• Fast and Light-Weight Network for Single Frame Structured Illumination Microscopy Super-Resolution [22.953512091536663]
  We propose a single-frame structured illumination microscopy (SF-SIM) based on deep learning. Our method is almost 14 times faster than traditional SIM methods when achieving similar results.
  arXiv  Detail & Related papers  (2021-11-17T13:39:41Z)
• Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation [60.663499381212425]
  We enhance the image quality by using a Wasserstein Generative Adversarial Network combined with a novel Adaptive Gradient Balancing technique. In MRI, our method minimizes artifacts, while maintaining a high-quality reconstruction that produces sharper images than other techniques.
  arXiv  Detail & Related papers  (2021-04-05T13:05:22Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.