MOSAIC: Masked Optimisation with Selective Attention for Image Reconstruction

• URL: http://arxiv.org/abs/2306.00906v1
• Date: Thu, 1 Jun 2023 17:05:02 GMT
• Title: MOSAIC: Masked Optimisation with Selective Attention for Image Reconstruction
• Authors: Pamuditha Somarathne, Tharindu Wickremasinghe, Amashi Niwarthana, A. Thieshanthan, Chamira U.S. Edussooriya, and Dushan N. Wadduwage
• Abstract summary: We propose a novel compressive sensing framework to reconstruct images given any random selection of measurements. MOSAIC incorporates an embedding technique to efficiently apply attention mechanisms on an encoded sequence of measurements. A range of experiments validate our proposed architecture as a promising alternative for existing CS reconstruction methods.
• Score: 0.5541644538483947
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compressive sensing (CS) reconstructs images from sub-Nyquist measurements by solving a sparsity-regularized inverse problem. Traditional CS solvers use iterative optimizers with hand crafted sparsifiers, while early data-driven methods directly learn an inverse mapping from the low-dimensional measurement space to the original image space. The latter outperforms the former, but is restrictive to a pre-defined measurement domain. More recent, deep unrolling methods combine traditional proximal gradient methods and data-driven approaches to iteratively refine an image approximation. To achieve higher accuracy, it has also been suggested to learn both the sampling matrix, and the choice of measurement vectors adaptively. Contrary to the current trend, in this work we hypothesize that a general inverse mapping from a random set of compressed measurements to the image domain exists for a given measurement basis, and can be learned. Such a model is single-shot, non-restrictive and does not parametrize the sampling process. To this end, we propose MOSAIC, a novel compressive sensing framework to reconstruct images given any random selection of measurements, sampled using a fixed basis. Motivated by the uneven distribution of information across measurements, MOSAIC incorporates an embedding technique to efficiently apply attention mechanisms on an encoded sequence of measurements, while dispensing the need to use unrolled deep networks. A range of experiments validate our proposed architecture as a promising alternative for existing CS reconstruction methods, by achieving the state-of-the-art for metrics of reconstruction accuracy on standard datasets.

Related papers

• Deep Learning Based Speckle Filtering for Polarimetric SAR Images. Application to Sentinel-1 [51.404644401997736]
  We propose a complete framework to remove speckle in polarimetric SAR images using a convolutional neural network. Experiments show that the proposed approach offers exceptional results in both speckle reduction and resolution preservation.
  arXiv  Detail & Related papers  (2024-08-28T10:07:17Z)
• Optimized Linear Measurements for Inverse Problems using Diffusion-Based Image Generation [10.297832938258841]
  We re-examine the problem of reconstructing a high-dimensional signal from a small set of linear measurements. We show that optimizing the measurements for the SSIM perceptual loss leads to perceptually improved reconstruction.
  arXiv  Detail & Related papers  (2024-05-22T20:38:58Z)
• Diffeomorphic Template Registration for Atmospheric Turbulence Mitigation [50.16004183320537]
  We describe a method for recovering the irradiance underlying a collection of images corrupted by atmospheric turbulence. We select one of the images as a reference, and model the deformation in this image by the aggregation of the optical flow from it to other images. We achieve state-of-the-art performance despite its simplicity.
  arXiv  Detail & Related papers  (2024-05-06T17:39:53Z)
• ReNoise: Real Image Inversion Through Iterative Noising [62.96073631599749]
  We introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models.
  arXiv  Detail & Related papers  (2024-03-21T17:52:08Z)
• MB-RACS: Measurement-Bounds-based Rate-Adaptive Image Compressed Sensing Network [65.1004435124796]
  We propose a Measurement-Bounds-based Rate-Adaptive Image Compressed Sensing Network (MB-RACS) framework. Our experiments demonstrate that the proposed MB-RACS method surpasses current leading methods.
  arXiv  Detail & Related papers  (2024-01-19T04:40:20Z)
• Posterior Estimation for Dynamic PET imaging using Conditional Variational Inference [10.206699988915183]
  We propose a deep-learning-based framework for efficient posterior estimation. Specifically, we counteract the information loss in the forward process by introducing latent variables.
  arXiv  Detail & Related papers  (2023-10-24T14:05:30Z)
• Fast Hybrid Image Retargeting [0.0]
  We propose a method that quantifies and limits warping distortions with the use of content-aware cropping. Our method outperforms recent approaches, while running in a fraction of their execution time.
  arXiv  Detail & Related papers  (2022-03-25T11:46:06Z)
• A Probabilistic Deep Image Prior for Computational Tomography [0.19573380763700707]
  Existing deep-learning based tomographic image reconstruction methods do not provide accurate estimates of reconstruction uncertainty. We construct a Bayesian prior for tomographic reconstruction, which combines the classical total variation (TV) regulariser with the modern deep image prior (DIP) For the inference, we develop an approach based on the linearised Laplace method, which is scalable to high-dimensional settings.
  arXiv  Detail & Related papers  (2022-02-28T14:47:14Z)
• Deblurring via Stochastic Refinement [85.42730934561101]
  We present an alternative framework for blind deblurring based on conditional diffusion models. Our method is competitive in terms of distortion metrics such as PSNR.
  arXiv  Detail & Related papers  (2021-12-05T04:36:09Z)
• Learned Block Iterative Shrinkage Thresholding Algorithm for Photothermal Super Resolution Imaging [52.42007686600479]
  We propose a learned block-sparse optimization approach using an iterative algorithm unfolded into a deep neural network. We show the benefits of using a learned block iterative shrinkage thresholding algorithm that is able to learn the choice of regularization parameters.
  arXiv  Detail & Related papers  (2020-12-07T09:27:16Z)
• Sampling possible reconstructions of undersampled acquisitions in MR imaging [9.75702493778194]
  Undersampling the k-space during MR saves time, however results in an ill-posed inversion problem, leading to an infinite set of images as possible solutions. Traditionally, this is tackled as a reconstruction problem by searching for a single "best" image out of this solution set according to some chosen regularization or prior. We propose a method that instead returns multiple images which are possible under the acquisition model and the chosen prior to capture the uncertainty in the inversion process.
  arXiv  Detail & Related papers  (2020-09-30T18:20:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.