Model Adaptation for Inverse Problems in Imaging

• URL: http://arxiv.org/abs/2012.00139v2
• Date: Mon, 12 Apr 2021 20:38:44 GMT
• Title: Model Adaptation for Inverse Problems in Imaging
• Authors: Davis Gilton, Gregory Ongie, Rebecca Willett
• Abstract summary: Deep neural networks have been applied successfully to a wide variety of inverse problems in imaging. We propose two novel procedures that adapt the network to a change in the forward model, even without full knowledge of the change. We show these simple model adaptation approaches achieve empirical success in a variety of inverse problems, including deblurring, super-resolution, and undersampled image reconstruction.
• Score: 14.945209750917483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks have been applied successfully to a wide variety of inverse problems arising in computational imaging. These networks are typically trained using a forward model that describes the measurement process to be inverted, which is often incorporated directly into the network itself. However, these approaches are sensitive to changes in the forward model: if at test time the forward model varies (even slightly) from the one the network was trained for, the reconstruction performance can degrade substantially. Given a network trained to solve an initial inverse problem with a known forward model, we propose two novel procedures that adapt the network to a change in the forward model, even without full knowledge of the change. Our approaches do not require access to more labeled data (i.e., ground truth images). We show these simple model adaptation approaches achieve empirical success in a variety of inverse problems, including deblurring, super-resolution, and undersampled image reconstruction in magnetic resonance imaging.

Related papers

• Imaging Signal Recovery Using Neural Network Priors Under Uncertain Forward Model Parameters [0.7724713939814069]
  Inverse imaging problems (IIPs) arise in various applications, with the main objective of reconstructing an image from its compressed measurements. We propose a novel Moment-Aggregation (MA) framework that is compatible with the popular IIP solution by using a neural network prior. We theoretically demonstrate the convergence of the MA framework, which has a similar complexity with reconstruction under the known forward model parameters.
  arXiv  Detail & Related papers  (2024-05-05T14:12:48Z)
• Mixture Manifold Networks: A Computationally Efficient Baseline for Inverse Modeling [7.891408798179181]
  We propose and show the efficacy of a new method to address generic inverse problems. Recent work has shown impressive results using deep learning, but we note that there is a trade-off between model performance and computational time.
  arXiv  Detail & Related papers  (2022-11-25T20:18:07Z)
• PatchNR: Learning from Small Data by Patch Normalizing Flow Regularization [57.37911115888587]
  We introduce a regularizer for the variational modeling of inverse problems in imaging based on normalizing flows. Our regularizer, called patchNR, involves a normalizing flow learned on patches of very few images.
  arXiv  Detail & Related papers  (2022-05-24T12:14:26Z)
• Convolutional Analysis Operator Learning by End-To-End Training of Iterative Neural Networks [3.6280929178575994]
  We show how convolutional sparsifying filters can be efficiently learned by end-to-end training of iterative neural networks. We evaluated our approach on a non-Cartesian 2D cardiac cine MRI example and show that the obtained filters are better suitable for the corresponding reconstruction algorithm than the ones obtained by decoupled pre-training.
  arXiv  Detail & Related papers  (2022-03-04T07:32:16Z)
• Training Adaptive Reconstruction Networks for Blind Inverse Problems [0.0]
  We show through various applications that training the network with a family of forward operators allows solving the adaptivity problem without compromising the reconstruction quality significantly. Experiments include partial Fourier sampling problems arising in magnetic resonance imaging (MRI) with sensitivity estimation and off-resonance effects, computerized tomography (CT) with a tilted geometry and image deblurring with Fresnel diffraction kernels.
  arXiv  Detail & Related papers  (2022-02-23T07:56:02Z)
• Denoising Diffusion Restoration Models [110.1244240726802]
  Denoising Diffusion Restoration Models (DDRM) is an efficient, unsupervised posterior sampling method. We demonstrate DDRM's versatility on several image datasets for super-resolution, deblurring, inpainting, and colorization.
  arXiv  Detail & Related papers  (2022-01-27T20:19:07Z)
• The Importance of the Current Input in Sequence Modeling [0.0]
  We show that a very simple idea, to add a direct connection between the input and the output, skipping the recurrent module, leads to an increase of the prediction accuracy. Experiments carried out on different problems show that the addition of this kind of connection to a recurrent network always improves the results.
  arXiv  Detail & Related papers  (2021-12-22T10:29:20Z)
• Deep Neural Networks are Surprisingly Reversible: A Baseline for Zero-Shot Inversion [90.65667807498086]
  This paper presents a zero-shot direct model inversion framework that recovers the input to the trained model given only the internal representation. We empirically show that modern classification models on ImageNet can, surprisingly, be inverted, allowing an approximate recovery of the original 224x224px images from a representation after more than 20 layers.
  arXiv  Detail & Related papers  (2021-07-13T18:01:43Z)
• SIR: Self-supervised Image Rectification via Seeing the Same Scene from Multiple Different Lenses [82.56853587380168]
  We propose a novel self-supervised image rectification (SIR) method based on an important insight that the rectified results of distorted images of the same scene from different lens should be the same. We leverage a differentiable warping module to generate the rectified images and re-distorted images from the distortion parameters. Our method achieves comparable or even better performance than the supervised baseline method and representative state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-30T08:23:25Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)
• On Robustness and Transferability of Convolutional Neural Networks [147.71743081671508]
  Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. We study the interplay between out-of-distribution and transfer performance of modern image classification CNNs for the first time. We find that increasing both the training set and model sizes significantly improve the distributional shift robustness.
  arXiv  Detail & Related papers  (2020-07-16T18:39:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.