Optimized Linear Measurements for Inverse Problems using Diffusion-Based Image Generation

• URL: http://arxiv.org/abs/2405.17456v1
• Date: Wed, 22 May 2024 20:38:58 GMT
• Title: Optimized Linear Measurements for Inverse Problems using Diffusion-Based Image Generation
• Authors: Ling-Qi Zhang, Zahra Kadkhodaie, Eero P. Simoncelli, David H. Brainard,
• Abstract summary: We re-examine the problem of reconstructing a high-dimensional signal from a small set of linear measurements. We show that optimizing the measurements for the SSIM perceptual loss leads to perceptually improved reconstruction.
• Score: 10.297832938258841
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We re-examine the problem of reconstructing a high-dimensional signal from a small set of linear measurements, in combination with image prior from a diffusion probabilistic model. Well-established methods for optimizing such measurements include principal component analysis (PCA), independent component analysis (ICA) and compressed sensing (CS), all of which rely on axis- or subspace-aligned statistical characterization. But many naturally occurring signals, including photographic images, contain richer statistical structure. To exploit such structure, we introduce a general method for obtaining an optimized set of linear measurements, assuming a Bayesian inverse solution that leverages the prior implicit in a neural network trained to perform denoising. We demonstrate that these measurements are distinct from those of PCA and CS, with significant improvements in minimizing squared reconstruction error. In addition, we show that optimizing the measurements for the SSIM perceptual loss leads to perceptually improved reconstruction. Our results highlight the importance of incorporating the specific statistical regularities of natural signals when designing effective linear measurements.

Related papers

• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation [57.10353686244835]
  We address the problem of verifying neural networks against geometric transformations of the input image, including rotation, scaling, shearing, and translation. The proposed method computes provably sound piecewise linear constraints for the pixel values by using sampling and linear approximations in combination with branch-and-bound Lipschitz. We show that our proposed implementation resolves up to 32% more verification cases than present approaches.
  arXiv  Detail & Related papers  (2024-08-23T15:02:09Z)
• Gradient Descent Provably Solves Nonlinear Tomographic Reconstruction [60.95625458395291]
  In computed tomography (CT) the forward model consists of a linear transform followed by an exponential nonlinearity based on the attenuation of light according to the Beer-Lambert Law. We show that this approach reduces metal artifacts compared to a commercial reconstruction of a human skull with metal crowns.
  arXiv  Detail & Related papers  (2023-10-06T00:47:57Z)
• Spectral Estimators for Structured Generalized Linear Models via Approximate Message Passing [28.91482208876914]
  We consider the problem of parameter estimation in a high-dimensional generalized linear model. Despite their wide use, a rigorous performance characterization, as well as a principled way to preprocess the data, are available only for unstructured designs.
  arXiv  Detail & Related papers  (2023-08-28T11:49:23Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• MOSAIC: Masked Optimisation with Selective Attention for Image Reconstruction [0.5541644538483947]
  We propose a novel compressive sensing framework to reconstruct images given any random selection of measurements. MOSAIC incorporates an embedding technique to efficiently apply attention mechanisms on an encoded sequence of measurements. A range of experiments validate our proposed architecture as a promising alternative for existing CS reconstruction methods.
  arXiv  Detail & Related papers  (2023-06-01T17:05:02Z)
• Amortized Bayesian Inference of GISAXS Data with Normalizing Flows [0.10752246796855561]
  We propose a simulation-based framework that combines variational auto-encoders and normalizing flows to estimate the posterior distribution of object parameters. We demonstrate that our method reduces the inference cost by orders of magnitude while producing consistent results with ABC.
  arXiv  Detail & Related papers  (2022-10-04T12:09:57Z)
• Design of Compressed Sensing Systems via Density-Evolution Framework for Structure Recovery in Graphical Models [10.667885727418705]
  It has been shown that learning the structure of Bayesian networks from observational data is an NP-Hard problem. We propose a novel density-evolution based framework for optimizing compressed linear measurement systems. We show that the structure of GBN can indeed be recovered from resulting compressed measurements.
  arXiv  Detail & Related papers  (2022-03-17T22:16:38Z)
• A Model for Multi-View Residual Covariances based on Perspective Deformation [88.21738020902411]
  We derive a model for the covariance of the visual residuals in multi-view SfM, odometry and SLAM setups. We validate our model with synthetic and real data and integrate it into photometric and feature-based Bundle Adjustment.
  arXiv  Detail & Related papers  (2022-02-01T21:21:56Z)
• Fast Scalable Image Restoration using Total Variation Priors and Expectation Propagation [7.7731951589289565]
  This paper presents a scalable approximate Bayesian method for image restoration using total variation (TV) priors. We use the expectation propagation (EP) framework to approximate minimum mean squared error (MMSE) estimators and marginal (pixel-wise) variances.
  arXiv  Detail & Related papers  (2021-10-04T17:28:41Z)
• Regularization by Denoising Sub-sampled Newton Method for Spectral CT Multi-Material Decomposition [78.37855832568569]
  We propose to solve a model-based maximum-a-posterior problem to reconstruct multi-materials images with application to spectral CT. In particular, we propose to solve a regularized optimization problem based on a plug-in image-denoising function. We show numerical and experimental results for spectral CT materials decomposition.
  arXiv  Detail & Related papers  (2021-03-25T15:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.