Differentiable Gaussianization Layers for Inverse Problems Regularized by Deep Generative Models

• URL: http://arxiv.org/abs/2112.03860v5
• Date: Mon, 29 Jul 2024 14:31:47 GMT
• Title: Differentiable Gaussianization Layers for Inverse Problems Regularized by Deep Generative Models
• Authors: Dongzhuo Li,
• Abstract summary: We show that latent tensors of deep generative models can fall out of the desired high-dimensional standard Gaussian distribution during inversion. Our approach achieves state-of-the-art performance in terms of accuracy and consistency.
• Score: 5.439020425819001
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep generative models such as GANs, normalizing flows, and diffusion models are powerful regularizers for inverse problems. They exhibit great potential for helping reduce ill-posedness and attain high-quality results. However, the latent tensors of such deep generative models can fall out of the desired high-dimensional standard Gaussian distribution during inversion, particularly in the presence of data noise and inaccurate forward models, leading to low-fidelity solutions. To address this issue, we propose to reparameterize and Gaussianize the latent tensors using novel differentiable data-dependent layers wherein custom operators are defined by solving optimization problems. These proposed layers constrain inverse problems to obtain high-fidelity in-distribution solutions. We validate our technique on three inversion tasks: compressive-sensing MRI, image deblurring, and eikonal tomography (a nonlinear PDE-constrained inverse problem) using two representative deep generative models: StyleGAN2 and Glow. Our approach achieves state-of-the-art performance in terms of accuracy and consistency.

Related papers

• G2D2: Gradient-guided Discrete Diffusion for image inverse problem solving [55.185588994883226]
  This paper presents a novel method for addressing linear inverse problems by leveraging image-generation models based on discrete diffusion as priors. To the best of our knowledge, this is the first approach to use discrete diffusion model-based priors for solving image inverse problems.
  arXiv  Detail & Related papers  (2024-10-09T06:18:25Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Enhancing Low-Order Discontinuous Galerkin Methods with Neural Ordinary Differential Equations for Compressible Navier--Stokes Equations [0.18648070031379424]
  It is common to run a low-fidelity model with a subgrid-scale model to reduce the computational cost. We propose a novel method for learning the subgrid-scale model effects when simulating partial differential equations augmented by neural ordinary differential operators. Our approach learns the missing scales of the low-order DG solver at a continuous level and hence improves the accuracy of the low-order DG approximations.
  arXiv  Detail & Related papers  (2023-10-29T04:26:23Z)
• Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency [7.671153315762146]
  Training diffusion models in the pixel space are both data-intensive and computationally demanding. Latent diffusion models, which operate in a much lower-dimensional space, offer a solution to these challenges. We propose textitReSample, an algorithm that can solve general inverse problems with pre-trained latent diffusion models.
  arXiv  Detail & Related papers  (2023-07-16T18:42:01Z)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models [101.831766524264]
  Inverse tasks can be formulated as inferring a posterior distribution over data. This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable. We propose a variational approach that by design seeks to approximate the true posterior distribution.
  arXiv  Detail & Related papers  (2023-05-07T23:00:47Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• Variational Laplace Autoencoders [53.08170674326728]
  Variational autoencoders employ an amortized inference model to approximate the posterior of latent variables. We present a novel approach that addresses the limited posterior expressiveness of fully-factorized Gaussian assumption. We also present a general framework named Variational Laplace Autoencoders (VLAEs) for training deep generative models.
  arXiv  Detail & Related papers  (2022-11-30T18:59:27Z)
• Intermediate Layer Optimization for Inverse Problems using Deep Generative Models [86.29330440222199]
  ILO is a novel optimization algorithm for solving inverse problems with deep generative models. We empirically show that our approach outperforms state-of-the-art methods introduced in StyleGAN-2 and PULSE for a wide range of inverse problems.
  arXiv  Detail & Related papers  (2021-02-15T06:52:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.