Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction

• URL: http://arxiv.org/abs/2402.04097v2
• Date: Thu, 8 Feb 2024 03:01:07 GMT
• Title: Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction
• Authors: Shijun Liang, Evan Bell, Qing Qu, Rongrong Wang, Saiprasad Ravishankar
• Abstract summary: We study how DIP recovers information from undersampled imaging measurements. We introduce a self-driven reconstruction process that concurrently optimize both the network weights and the input. Our method incorporates a novel denoiser regularization term which enables robust and stable joint estimation of both the network input and reconstructed image.
• Score: 13.277067849874756
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ability of deep image prior (DIP) to recover high-quality images from incomplete or corrupted measurements has made it popular in inverse problems in image restoration and medical imaging including magnetic resonance imaging (MRI). However, conventional DIP suffers from severe overfitting and spectral bias effects. In this work, we first provide an analysis of how DIP recovers information from undersampled imaging measurements by analyzing the training dynamics of the underlying networks in the kernel regime for different architectures. This study sheds light on important underlying properties for DIP-based recovery. Current research suggests that incorporating a reference image as network input can enhance DIP's performance in image reconstruction compared to using random inputs. However, obtaining suitable reference images requires supervision, and raises practical difficulties. In an attempt to overcome this obstacle, we further introduce a self-driven reconstruction process that concurrently optimizes both the network weights and the input while eliminating the need for training data. Our method incorporates a novel denoiser regularization term which enables robust and stable joint estimation of both the network input and reconstructed image. We demonstrate that our self-guided method surpasses both the original DIP and modern supervised methods in terms of MR image reconstruction performance and outperforms previous DIP-based schemes for image inpainting.

Related papers

• DensePANet: An improved generative adversarial network for photoacoustic tomography image reconstruction from sparse data [1.4665304971699265]
  We propose an end-to-end method called DensePANet to solve the problem of PAT image reconstruction from sparse data. The proposed model employs a novel modification of UNet in its generator, called FD-UNet++, which considerably improves the reconstruction performance.
  arXiv  Detail & Related papers  (2024-04-19T09:52:32Z)
• Enhancing Low-dose CT Image Reconstruction by Integrating Supervised and Unsupervised Learning [13.17680480211064]
  We propose a hybrid supervised-unsupervised learning framework for X-ray computed tomography (CT) image reconstruction. Each proposed trained block consists of a deterministic MBIR solver and a neural network. We demonstrate the efficacy of this learned hybrid model for low-dose CT image reconstruction with limited training data.
  arXiv  Detail & Related papers  (2023-11-19T20:23:59Z)
• Uncertainty-Aware Null Space Networks for Data-Consistent Image Reconstruction [0.0]
  State-of-the-art reconstruction methods have been developed based on recent advances in deep learning. For such approaches to be used in safety-critical domains such as medical imaging, the network reconstruction should not only provide the user with a reconstructed image, but also with some level of confidence in the reconstruction. This work is the first approach to solving inverse problems that additionally models data-dependent uncertainty by estimating an input-dependent scale map.
  arXiv  Detail & Related papers  (2023-04-14T06:58:44Z)
• Residual Back Projection With Untrained Neural Networks [1.2707050104493216]
  We present a framework for iterative reconstruction (IR) in computed tomography (CT) Our framework incorporates this structural information as a deep image prior (DIP) We propose using an untrained U-net in conjunction with a novel residual back projection to minimize an objective function and achieve high-accuracy reconstruction.
  arXiv  Detail & Related papers  (2022-10-26T01:58:09Z)
• Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction [68.80715727288514]
  We show how to unfold an iterative MGDUN algorithm into a novel model-guided deep unfolding network by taking the MRI observation matrix. In this paper, we propose a novel Model-Guided interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction.
  arXiv  Detail & Related papers  (2022-09-15T03:58:30Z)
• A Long Short-term Memory Based Recurrent Neural Network for Interventional MRI Reconstruction [50.1787181309337]
  We propose a convolutional long short-term memory (Conv-LSTM) based recurrent neural network (RNN), or ConvLR, to reconstruct interventional images with golden-angle radial sampling. The proposed algorithm has the potential to achieve real-time i-MRI for DBS and can be used for general purpose MR-guided intervention.
  arXiv  Detail & Related papers  (2022-03-28T14:03:45Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Multi-modal Aggregation Network for Fast MR Imaging [85.25000133194762]
  We propose a novel Multi-modal Aggregation Network, named MANet, which is capable of discovering complementary representations from a fully sampled auxiliary modality. In our MANet, the representations from the fully sampled auxiliary and undersampled target modalities are learned independently through a specific network. Our MANet follows a hybrid domain learning framework, which allows it to simultaneously recover the frequency signal in the $k$-space domain.
  arXiv  Detail & Related papers  (2021-10-15T13:16:59Z)
• Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation [60.663499381212425]
  We enhance the image quality by using a Wasserstein Generative Adversarial Network combined with a novel Adaptive Gradient Balancing technique. In MRI, our method minimizes artifacts, while maintaining a high-quality reconstruction that produces sharper images than other techniques.
  arXiv  Detail & Related papers  (2021-04-05T13:05:22Z)
• Joint reconstruction and bias field correction for undersampled MR imaging [7.409376558513677]
  Undersampling the k-space in MRI allows saving precious acquisition time, yet results in an ill-posed inversion problem. Deep learning schemes are susceptible to differences between the training data and the image to be reconstructed at test time. In this work, we address the sensitivity of the reconstruction problem to the bias field and propose to model it explicitly in the reconstruction.
  arXiv  Detail & Related papers  (2020-07-26T12:58:34Z)
• Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation [92.86123832948809]
  We propose a deep face super-resolution (FSR) method with iterative collaboration between two recurrent networks. In each recurrent step, the recovery branch utilizes the prior knowledge of landmarks to yield higher-quality images. A new attentive fusion module is designed to strengthen the guidance of landmark maps.
  arXiv  Detail & Related papers  (2020-03-29T16:04:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.