On the Robustness of Normalizing Flows for Inverse Problems in Imaging
- URL: http://arxiv.org/abs/2212.04319v1
- Date: Thu, 8 Dec 2022 15:18:28 GMT
- Title: On the Robustness of Normalizing Flows for Inverse Problems in Imaging
- Authors: Seongmin Hong, Inbum Park, Se Young Chun
- Abstract summary: Unintended severe artifacts are occasionally observed in the output of Conditional normalizing flows.
We empirically and theoretically reveal that these problems are caused by exploding variance'' in the conditional affine coupling layer.
We suggest a simple remedy that substitutes the affine coupling layers with the modified rational quadratic spline coupling layers in normalizing flows.
- Score: 16.18759484251522
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Conditional normalizing flows can generate diverse image samples for solving
inverse problems. Most normalizing flows for inverse problems in imaging employ
the conditional affine coupling layer that can generate diverse images quickly.
However, unintended severe artifacts are occasionally observed in the output of
them. In this work, we address this critical issue by investigating the origins
of these artifacts and proposing the conditions to avoid them. First of all, we
empirically and theoretically reveal that these problems are caused by
``exploding variance'' in the conditional affine coupling layer for certain
out-of-distribution (OOD) conditional inputs. Then, we further validated that
the probability of causing erroneous artifacts in pixels is highly correlated
with a Mahalanobis distance-based OOD score for inverse problems in imaging.
Lastly, based on our investigations, we propose a remark to avoid exploding
variance and then based on it, we suggest a simple remedy that substitutes the
affine coupling layers with the modified rational quadratic spline coupling
layers in normalizing flows, to encourage the robustness of generated image
samples. Our experimental results demonstrated that our suggested methods
effectively suppressed critical artifacts occurring in normalizing flows for
super-resolution space generation and low-light image enhancement without
compromising performance.
Related papers
- pcaGAN: Improving Posterior-Sampling cGANs via Principal Component Regularization [11.393603788068777]
In ill-posed imaging inverse problems, there can exist many hypotheses that fit both the observed measurements and prior knowledge of the true image.
We propose a fast and accurate posterior-sampling conditional generative adversarial network (cGAN) that, through a novel form of regularization, aims for correctness in the posterior mean.
arXiv Detail & Related papers (2024-11-01T14:09:28Z) - FlowDepth: Decoupling Optical Flow for Self-Supervised Monocular Depth Estimation [8.78717459496649]
We propose FlowDepth, where a Dynamic Motion Flow Module (DMFM) decouples the optical flow by a mechanism-based approach and warps the dynamic regions thus solving the mismatch problem.
For the unfairness of photometric errors caused by high-freq and low-texture regions, we use Depth-Cue-Aware Blur (DCABlur) and Cost-Volume sparsity loss respectively at the input and the loss level to solve the problem.
arXiv Detail & Related papers (2024-03-28T10:31:23Z) - A Variational Perspective on Solving Inverse Problems with Diffusion
Models [101.831766524264]
Inverse tasks can be formulated as inferring a posterior distribution over data.
This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable.
We propose a variational approach that by design seeks to approximate the true posterior distribution.
arXiv Detail & Related papers (2023-05-07T23:00:47Z) - JPEG Artifact Correction using Denoising Diffusion Restoration Models [110.1244240726802]
We build upon Denoising Diffusion Restoration Models (DDRM) and propose a method for solving some non-linear inverse problems.
We leverage the pseudo-inverse operator used in DDRM and generalize this concept for other measurement operators.
arXiv Detail & Related papers (2022-09-23T23:47:00Z) - Improving Diffusion Models for Inverse Problems using Manifold Constraints [55.91148172752894]
We show that current solvers throw the sample path off the data manifold, and hence the error accumulates.
To address this, we propose an additional correction term inspired by the manifold constraint.
We show that our method is superior to the previous methods both theoretically and empirically.
arXiv Detail & Related papers (2022-06-02T09:06:10Z) - Normalizing Flow as a Flexible Fidelity Objective for Photo-Realistic
Super-resolution [161.39504409401354]
Super-resolution is an ill-posed problem, where a ground-truth high-resolution image represents only one possibility in the space of plausible solutions.
Yet, the dominant paradigm is to employ pixel-wise losses, such as L_, which drive the prediction towards a blurry average.
We address this issue by revisiting the L_ loss and show that it corresponds to a one-layer conditional flow.
Inspired by this relation, we explore general flows as a fidelity-based alternative to the L_ objective.
We demonstrate that the flexibility of deeper flows leads to better visual quality and consistency when combined with adversarial losses.
arXiv Detail & Related papers (2021-11-05T17:56:51Z) - Generative Flows as a General Purpose Solution for Inverse Problems [0.0]
We propose a regularization term to directly produce high likelihood reconstructions.
We evaluate our method in image denoising, image deblurring, image inpainting, and image colorization.
arXiv Detail & Related papers (2021-10-25T21:56:44Z) - Low-Light Image Enhancement with Normalizing Flow [92.52290821418778]
In this paper, we investigate to model this one-to-many relationship via a proposed normalizing flow model.
An invertible network that takes the low-light images/features as the condition and learns to map the distribution of normally exposed images into a Gaussian distribution.
The experimental results on the existing benchmark datasets show our method achieves better quantitative and qualitative results, obtaining better-exposed illumination, less noise and artifact, and richer colors.
arXiv Detail & Related papers (2021-09-13T12:45:08Z) - On Measuring and Controlling the Spectral Bias of the Deep Image Prior [63.88575598930554]
The deep image prior has demonstrated the remarkable ability that untrained networks can address inverse imaging problems.
It requires an oracle to determine when to stop the optimization as the performance degrades after reaching a peak.
We study the deep image prior from a spectral bias perspective to address these problems.
arXiv Detail & Related papers (2021-07-02T15:10:42Z) - Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser [7.7288480250888]
We develop a robust and general methodology for making use of implicit priors in deep neural networks.
A CNN trained to perform blind (i.e., with unknown noise level) least-squares denoising is presented.
A generalization of this algorithm to constrained sampling provides a method for using the implicit prior to solve any linear inverse problem.
arXiv Detail & Related papers (2020-07-27T15:40:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.