Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy

• URL: http://arxiv.org/abs/2103.14015v1
• Date: Thu, 25 Mar 2021 17:47:02 GMT
• Title: Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy
• Authors: Agnieszka Barbara Szczotka, Dzhoshkun Ismail Shakir, Matthew J. Clarkson, Stephen P. Pereira, Tom Vercauteren
• Abstract summary: Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs) Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth high-resolution (HR) images. We design a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner.
• Score: 5.540381950806677
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Super-resolution (SR) methods have seen significant advances thanks to the development of convolutional neural networks (CNNs). CNNs have been successfully employed to improve the quality of endomicroscopy imaging. Yet, the inherent limitation of research on SR in endomicroscopy remains the lack of ground truth high-resolution (HR) images, commonly used for both supervised training and reference-based image quality assessment (IQA). Therefore, alternative methods, such as unsupervised SR are being explored. To address the need for non-reference image quality improvement, we designed a novel zero-shot super-resolution (ZSSR) approach that relies only on the endomicroscopy data to be processed in a self-supervised manner without the need for ground-truth HR images. We tailored the proposed pipeline to the idiosyncrasies of endomicroscopy by introducing both: a physically-motivated Voronoi downscaling kernel accounting for the endomicroscope's irregular fibre-based sampling pattern, and realistic noise patterns. We also took advantage of video sequences to exploit a sequence of images for self-supervised zero-shot image quality improvement. We run ablation studies to assess our contribution in regards to the downscaling kernel and noise simulation. We validate our methodology on both synthetic and original data. Synthetic experiments were assessed with reference-based IQA, while our results for original images were evaluated in a user study conducted with both expert and non-expert observers. The results demonstrated superior performance in image quality of ZSSR reconstructions in comparison to the baseline method. The ZSSR is also competitive when compared to supervised single-image SR, especially being the preferred reconstruction technique by experts.

Related papers

• On the Foundation Model for Cardiac MRI Reconstruction [6.284878525302227]
  We propose a foundation model that uses adaptive unrolling, channel-shifting, and Pattern and Contrast-Prompt-UNet to tackle the problem. The PCP-UNet is equipped with an image contrast and sampling pattern prompt.
  arXiv  Detail & Related papers  (2024-11-15T18:15:56Z)
• Rotational Augmented Noise2Inverse for Low-dose Computed Tomography Reconstruction [83.73429628413773]
  Supervised deep learning methods have shown the ability to remove noise in images but require accurate ground truth. We propose a novel self-supervised framework for LDCT, in which ground truth is not required for training the convolutional neural network (CNN) Numerical and experimental results show that the reconstruction accuracy of N2I with sparse views is degrading while the proposed rotational augmented Noise2Inverse (RAN2I) method keeps better image quality over a different range of sampling angles.
  arXiv  Detail & Related papers  (2023-12-19T22:40:51Z)
• Dual-Domain Self-Supervised Learning for Accelerated Non-Cartesian MRI Reconstruction [14.754843942604472]
  We present a fully self-supervised approach for accelerated non-Cartesian MRI reconstruction. In training, the undersampled data are split into disjoint k-space domain partitions. For the image-level self-supervision, we enforce appearance consistency obtained from the original undersampled data.
  arXiv  Detail & Related papers  (2023-02-18T06:11:49Z)
• Stable Deep MRI Reconstruction using Generative Priors [13.400444194036101]
  We propose a novel deep neural network based regularizer which is trained in a generative setting on reference magnitude images only. The results demonstrate competitive performance, on par with state-of-the-art end-to-end deep learning methods.
  arXiv  Detail & Related papers  (2022-10-25T08:34:29Z)
• Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction [68.80715727288514]
  We show how to unfold an iterative MGDUN algorithm into a novel model-guided deep unfolding network by taking the MRI observation matrix. In this paper, we propose a novel Model-Guided interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction.
  arXiv  Detail & Related papers  (2022-09-15T03:58:30Z)
• Two-Stage Self-Supervised Cycle-Consistency Network for Reconstruction of Thin-Slice MR Images [62.4428833931443]
  The thick-slice magnetic resonance (MR) images are often structurally blurred in coronal and sagittal views. Deep learning has shown great potential to re-construct the high-resolution (HR) thin-slice MR images from those low-resolution (LR) cases. We propose a novel Two-stage Self-supervised Cycle-consistency Network (TSCNet) for MR slice reconstruction.
  arXiv  Detail & Related papers  (2021-06-29T13:29:18Z)
• Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation [60.663499381212425]
  We enhance the image quality by using a Wasserstein Generative Adversarial Network combined with a novel Adaptive Gradient Balancing technique. In MRI, our method minimizes artifacts, while maintaining a high-quality reconstruction that produces sharper images than other techniques.
  arXiv  Detail & Related papers  (2021-04-05T13:05:22Z)
• Perception Consistency Ultrasound Image Super-resolution via Self-supervised CycleGAN [63.49373689654419]
  We propose a new perception consistency ultrasound image super-resolution (SR) method based on self-supervision and cycle generative adversarial network (CycleGAN) We first generate the HR fathers and the LR sons of the test ultrasound LR image through image enhancement. We then make full use of the cycle loss of LR-SR-LR and HR-LR-SR and the adversarial characteristics of the discriminator to promote the generator to produce better perceptually consistent SR results.
  arXiv  Detail & Related papers  (2020-12-28T08:24:04Z)
• Joint reconstruction and bias field correction for undersampled MR imaging [7.409376558513677]
  Undersampling the k-space in MRI allows saving precious acquisition time, yet results in an ill-posed inversion problem. Deep learning schemes are susceptible to differences between the training data and the image to be reconstructed at test time. In this work, we address the sensitivity of the reconstruction problem to the bias field and propose to model it explicitly in the reconstruction.
  arXiv  Detail & Related papers  (2020-07-26T12:58:34Z)
• Improved Slice-wise Tumour Detection in Brain MRIs by Computing Dissimilarities between Latent Representations [68.8204255655161]
  Anomaly detection for Magnetic Resonance Images (MRIs) can be solved with unsupervised methods. We have proposed a slice-wise semi-supervised method for tumour detection based on the computation of a dissimilarity function in the latent space of a Variational AutoEncoder. We show that by training the models on higher resolution images and by improving the quality of the reconstructions, we obtain results which are comparable with different baselines.
  arXiv  Detail & Related papers  (2020-07-24T14:02:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.