Deep Nonparametric Convexified Filtering for Computational Photography, Image Synthesis and Adversarial Defense

• URL: http://arxiv.org/abs/2309.06724v2
• Date: Thu, 14 Sep 2023 02:44:57 GMT
• Title: Deep Nonparametric Convexified Filtering for Computational Photography, Image Synthesis and Adversarial Defense
• Authors: Jianqiao Wangni
• Abstract summary: We aim to provide a general framework for computational photography that recovers the real scene from imperfect images. It is consists of a nonparametric deep network to resemble the physical equations behind the image formation. We empirically verify its capability to defend image classification deep networks against adversary attack algorithms in real-time.
• Score: 1.79487674052027
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We aim to provide a general framework of for computational photography that recovers the real scene from imperfect images, via the Deep Nonparametric Convexified Filtering (DNCF). It is consists of a nonparametric deep network to resemble the physical equations behind the image formation, such as denoising, super-resolution, inpainting, and flash. DNCF has no parameterization dependent on training data, therefore has a strong generalization and robustness to adversarial image manipulation. During inference, we also encourage the network parameters to be nonnegative and create a bi-convex function on the input and parameters, and this adapts to second-order optimization algorithms with insufficient running time, having 10X acceleration over Deep Image Prior. With these tools, we empirically verify its capability to defend image classification deep networks against adversary attack algorithms in real-time.

Related papers

• Chasing Better Deep Image Priors between Over- and Under-parameterization [63.8954152220162]
  We study a novel "lottery image prior" (LIP) by exploiting DNN inherent sparsity. LIPworks significantly outperform deep decoders under comparably compact model sizes. We also extend LIP to compressive sensing image reconstruction, where a pre-trained GAN generator is used as the prior.
  arXiv  Detail & Related papers  (2024-10-31T17:49:44Z)
• PRISTA-Net: Deep Iterative Shrinkage Thresholding Network for Coded Diffraction Patterns Phase Retrieval [6.982256124089]
  Phase retrieval is a challenge nonlinear inverse problem in computational imaging and image processing. We have developed PRISTA-Net, a deep unfolding network based on the first-order iterative threshold threshold algorithm (ISTA) All parameters in the proposed PRISTA-Net framework, including the nonlinear transformation, threshold, and step size, are learned-to-end instead of being set.
  arXiv  Detail & Related papers  (2023-09-08T07:37:15Z)
• Deep Dynamic Scene Deblurring from Optical Flow [53.625999196063574]
  Deblurring can provide visually more pleasant pictures and make photography more convenient. It is difficult to model the non-uniform blur mathematically. We develop a convolutional neural network (CNN) to restore the sharp images from the deblurred features.
  arXiv  Detail & Related papers  (2023-01-18T06:37:21Z)
• DELAD: Deep Landweber-guided deconvolution with Hessian and sparse prior [0.22940141855172028]
  We present a model for non-blind image deconvolution that incorporates the classic iterative method into a deep learning application. We build our network based on the iterative Landweber deconvolution algorithm, which is integrated with trainable convolutional layers to enhance the recovered image structures and details.
  arXiv  Detail & Related papers  (2022-09-30T11:15:03Z)
• On Measuring and Controlling the Spectral Bias of the Deep Image Prior [63.88575598930554]
  The deep image prior has demonstrated the remarkable ability that untrained networks can address inverse imaging problems. It requires an oracle to determine when to stop the optimization as the performance degrades after reaching a peak. We study the deep image prior from a spectral bias perspective to address these problems.
  arXiv  Detail & Related papers  (2021-07-02T15:10:42Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• Real-Time, Deep Synthetic Aperture Sonar (SAS) Autofocus [34.77467193499518]
  Synthetic aperture sonar (SAS) requires precise time-of-flight measurements of the transmitted/received waveform to produce well-focused imagery. To overcome this, an emphautofocus algorithm is employed as a post-processing step after image reconstruction to improve image focus. We propose a deep learning technique to overcome these limitations and implicitly learn the weighting function in a data-driven manner.
  arXiv  Detail & Related papers  (2021-03-18T15:16:29Z)
• Adversarial Imaging Pipelines [28.178120782659878]
  We develop an attack that deceives a specific camera ISP while leaving others intact. We validate the proposed method using recent state-of-the-art automotive hardware ISPs.
  arXiv  Detail & Related papers  (2021-02-07T06:10:54Z)
• Adversarial Robustness Across Representation Spaces [35.58913661509278]
  Adversa robustness corresponds to the susceptibility of deep neural networks to imperceptible perturbations made at test time. In this work we extend the setting to consider the problem of training of deep neural networks that can be made simultaneously robust to perturbations applied in multiple natural representation spaces.
  arXiv  Detail & Related papers  (2020-12-01T19:55:58Z)
• Perceptually Optimizing Deep Image Compression [53.705543593594285]
  Mean squared error (MSE) and $ell_p$ norms have largely dominated the measurement of loss in neural networks. We propose a different proxy approach to optimize image analysis networks against quantitative perceptual models.
  arXiv  Detail & Related papers  (2020-07-03T14:33:28Z)
• BP-DIP: A Backprojection based Deep Image Prior [49.375539602228415]
  We propose two image restoration approaches: (i) Deep Image Prior (DIP), which trains a convolutional neural network (CNN) from scratch in test time using the degraded image; and (ii) a backprojection (BP) fidelity term, which is an alternative to the standard least squares loss that is usually used in previous DIP works. We demonstrate the performance of the proposed method, termed BP-DIP, on the deblurring task and show its advantages over the plain DIP, with both higher PSNR values and better inference run-time.
  arXiv  Detail & Related papers  (2020-03-11T17:09:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.