The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior

• URL: http://arxiv.org/abs/2304.11409v2
• Date: Sun, 27 Aug 2023 00:50:26 GMT
• Title: The Devil is in the Upsampling: Architectural Decisions Made Simpler for Denoising with Deep Image Prior
• Authors: Yilin Liu, Jiang Li, Yunkui Pang, Dong Nie, Pew-thian Yap
• Abstract summary: Deep Image Prior (DIP) shows that some network architectures naturally bias towards smooth images and resist noises. Although DIP has removed the requirement of large training sets, it still presents two practical challenges for denoising: architectural design and noise-fitting. In this study, we analyze from a frequency perspective to demonstrate that the unlearnt upsampling is the main driving force behind the denoising phenomenon in DIP.
• Score: 20.1435183909115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Image Prior (DIP) shows that some network architectures naturally bias towards smooth images and resist noises, a phenomenon known as spectral bias. Image denoising is an immediate application of this property. Although DIP has removed the requirement of large training sets, it still presents two practical challenges for denoising: architectural design and noise-fitting, which are often intertwined. Existing methods mostly handcraft or search for the architecture from a large design space, due to the lack of understanding on how the architectural choice corresponds to the image. In this study, we analyze from a frequency perspective to demonstrate that the unlearnt upsampling is the main driving force behind the denoising phenomenon in DIP. This finding then leads to strategies for estimating a suitable architecture for every image without a laborious search. Extensive experiments show that the estimated architectures denoise and preserve the textural details better than current methods with up to 95% fewer parameters. The under-parameterized nature also makes them especially robust to a higher level of noise.

Related papers

• Masked Image Training for Generalizable Deep Image Denoising [53.03126421917465]
  We present a novel approach to enhance the generalization performance of denoising networks. Our method involves masking random pixels of the input image and reconstructing the missing information during training. Our approach exhibits better generalization ability than other deep learning models and is directly applicable to real-world scenarios.
  arXiv  Detail & Related papers  (2023-03-23T09:33:44Z)
• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Zero-shot Blind Image Denoising via Implicit Neural Representations [77.79032012459243]
  We propose an alternative denoising strategy that leverages the architectural inductive bias of implicit neural representations (INRs) We show that our method outperforms existing zero-shot denoising methods under an extensive set of low-noise or real-noise scenarios.
  arXiv  Detail & Related papers  (2022-04-05T12:46:36Z)
• Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis [148.16279746287452]
  We propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise. Experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-24T18:11:31Z)
• Physics-based Noise Modeling for Extreme Low-light Photography [63.65570751728917]
  We study the noise statistics in the imaging pipeline of CMOS photosensors. We formulate a comprehensive noise model that can accurately characterize the real noise structures. Our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms.
  arXiv  Detail & Related papers  (2021-08-04T16:36:29Z)
• Deep learning architectural designs for super-resolution of noisy images [7.657378889055478]
  In this work, we propose to jointly perform denoising and super-resolution. We investigate two architectural designs: "in-network" combines both tasks at feature level, while "pre-network" first performs denoising and then super-resolution. The pre-network design exhibits superior performance on unseen types of image corruption, which is a pathological failure case of existing super-resolution models.
  arXiv  Detail & Related papers  (2021-02-09T20:09:42Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)
• Reconstructing the Noise Manifold for Image Denoising [56.562855317536396]
  We introduce the idea of a cGAN which explicitly leverages structure in the image noise space. By learning directly a low dimensional manifold of the image noise, the generator promotes the removal from the noisy image only that information which spans this manifold. Based on our experiments, our model substantially outperforms existing state-of-the-art architectures.
  arXiv  Detail & Related papers  (2020-02-11T00:31:31Z)
• Noise2Inverse: Self-supervised deep convolutional denoising for tomography [0.0]
  Noise2Inverse is a deep CNN-based denoising method for linear image reconstruction algorithms. We develop a theoretical framework which shows that such training indeed obtains a denoising CNN. On simulated CT datasets, Noise2Inverse demonstrates an improvement in peak signal-to-noise ratio and structural similarity index.
  arXiv  Detail & Related papers  (2020-01-31T12:50:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.