Chasing Better Deep Image Priors between Over- and Under-parameterization

• URL: http://arxiv.org/abs/2410.24187v1
• Date: Thu, 31 Oct 2024 17:49:44 GMT
• Title: Chasing Better Deep Image Priors between Over- and Under-parameterization
• Authors: Qiming Wu, Xiaohan Chen, Yifan Jiang, Zhangyang Wang,
• Abstract summary: We study a novel "lottery image prior" (LIP) by exploiting DNN inherent sparsity. LIPworks significantly outperform deep decoders under comparably compact model sizes. We also extend LIP to compressive sensing image reconstruction, where a pre-trained GAN generator is used as the prior.
• Score: 63.8954152220162
• License:
• Abstract: Deep Neural Networks (DNNs) are well-known to act as over-parameterized deep image priors (DIP) that regularize various image inverse problems. Meanwhile, researchers also proposed extremely compact, under-parameterized image priors (e.g., deep decoder) that are strikingly competent for image restoration too, despite a loss of accuracy. These two extremes push us to think whether there exists a better solution in the middle: between over- and under-parameterized image priors, can one identify "intermediate" parameterized image priors that achieve better trade-offs between performance, efficiency, and even preserving strong transferability? Drawing inspirations from the lottery ticket hypothesis (LTH), we conjecture and study a novel "lottery image prior" (LIP) by exploiting DNN inherent sparsity, stated as: given an over-parameterized DNN-based image prior, it will contain a sparse subnetwork that can be trained in isolation, to match the original DNN's performance when being applied as a prior to various image inverse problems. Our results validate the superiority of LIPs: we can successfully locate the LIP subnetworks from over-parameterized DIPs at substantial sparsity ranges. Those LIP subnetworks significantly outperform deep decoders under comparably compact model sizes (by often fully preserving the effectiveness of their over-parameterized counterparts), and they also possess high transferability across different images as well as restoration task types. Besides, we also extend LIP to compressive sensing image reconstruction, where a pre-trained GAN generator is used as the prior (in contrast to untrained DIP or deep decoder), and confirm its validity in this setting too. To our best knowledge, this is the first time that LTH is demonstrated to be relevant in the context of inverse problems or image priors.

Related papers

• Blind Image Deconvolution Using Variational Deep Image Prior [4.92175281564179]
  This paper proposes a new variational deep image prior (VDIP) for blind image deconvolution. VDIP exploits additive hand-crafted image priors on latent sharp images and approximates a distribution for each pixel to avoid suboptimal solutions. Experiments show that the generated images have better quality than that of the original DIP on benchmark datasets.
  arXiv  Detail & Related papers  (2022-02-01T01:33:58Z)
• On Measuring and Controlling the Spectral Bias of the Deep Image Prior [63.88575598930554]
  The deep image prior has demonstrated the remarkable ability that untrained networks can address inverse imaging problems. It requires an oracle to determine when to stop the optimization as the performance degrades after reaching a peak. We study the deep image prior from a spectral bias perspective to address these problems.
  arXiv  Detail & Related papers  (2021-07-02T15:10:42Z)
• Image Restoration by Deep Projected GSURE [115.57142046076164]
  Ill-posed inverse problems appear in many image processing applications, such as deblurring and super-resolution. We propose a new image restoration framework that is based on minimizing a loss function that includes a "projected-version" of the Generalized SteinUnbiased Risk Estimator (GSURE) and parameterization of the latent image by a CNN.
  arXiv  Detail & Related papers  (2021-02-04T08:52:46Z)
• Blind Image Restoration with Flow Based Priors [19.190289348734215]
  In a blind setting with unknown degradations, a good prior remains crucial. We propose using normalizing flows to model the distribution of the target content and to use this as a prior in a maximum a posteriori (MAP) formulation. To the best of our knowledge, this is the first work that explores normalizing flows as prior in image enhancement problems.
  arXiv  Detail & Related papers  (2020-09-09T21:40:11Z)
• The Power of Triply Complementary Priors for Image Compressive Sensing [89.14144796591685]
  We propose a joint low-rank deep (LRD) image model, which contains a pair of complementaryly trip priors. We then propose a novel hybrid plug-and-play framework based on the LRD model for image CS. To make the optimization tractable, a simple yet effective algorithm is proposed to solve the proposed H-based image CS problem.
  arXiv  Detail & Related papers  (2020-05-16T08:17:44Z)
• Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation [181.08127307338654]
  This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. The deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images.
  arXiv  Detail & Related papers  (2020-03-30T17:45:07Z)
• BP-DIP: A Backprojection based Deep Image Prior [49.375539602228415]
  We propose two image restoration approaches: (i) Deep Image Prior (DIP), which trains a convolutional neural network (CNN) from scratch in test time using the degraded image; and (ii) a backprojection (BP) fidelity term, which is an alternative to the standard least squares loss that is usually used in previous DIP works. We demonstrate the performance of the proposed method, termed BP-DIP, on the deblurring task and show its advantages over the plain DIP, with both higher PSNR values and better inference run-time.
  arXiv  Detail & Related papers  (2020-03-11T17:09:12Z)
• Reducing the Representation Error of GAN Image Priors Using the Deep Decoder [29.12824512060469]
  We show a method for reducing the representation error of GAN priors by modeling images as the linear combination of a GAN prior and a Deep Decoder. For compressive sensing and image superresolution, our hybrid model exhibits consistently higher PSNRs than both the GAN priors and Deep Decoder separately.
  arXiv  Detail & Related papers  (2020-01-23T18:37:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.