Snapshot Compressive Imaging: Principle, Implementation, Theory, Algorithms and Applications

• URL: http://arxiv.org/abs/2103.04421v1
• Date: Sun, 7 Mar 2021 18:31:47 GMT
• Title: Snapshot Compressive Imaging: Principle, Implementation, Theory, Algorithms and Applications
• Authors: Xin Yuan and David J. Brady and Aggelos K. Katsaggelos
• Abstract summary: Snapshot compressive imaging (SCI) uses a two-dimensional (2D) detector to capture HD ($ge3$D) data in a em snapshot measurement. This article reviews recent advances in SCI hardware, theory and algorithms, including both optimization-based and deep-learning-based algorithms.
• Score: 23.304009992886286
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Capturing high-dimensional (HD) data is a long-term challenge in signal processing and related fields. Snapshot compressive imaging (SCI) uses a two-dimensional (2D) detector to capture HD ($\ge3$D) data in a {\em snapshot} measurement. Via novel optical designs, the 2D detector samples the HD data in a {\em compressive} manner; following this, algorithms are employed to reconstruct the desired HD data-cube. SCI has been used in hyperspectral imaging, video, holography, tomography, focal depth imaging, polarization imaging, microscopy, \etc.~Though the hardware has been investigated for more than a decade, the theoretical guarantees have only recently been derived. Inspired by deep learning, various deep neural networks have also been developed to reconstruct the HD data-cube in spectral SCI and video SCI. This article reviews recent advances in SCI hardware, theory and algorithms, including both optimization-based and deep-learning-based algorithms. Diverse applications and the outlook of SCI are also discussed.

Related papers

• Deep Optics for Video Snapshot Compressive Imaging [10.830072985735175]
  Video snapshot imaging (SCI) aims to capture a sequence of video frames with only a single shot of a 2D detector. This paper presents a framework to jointly optimize masks and a reconstruction network. We believe this is a milestone for real-world video SCI.
  arXiv  Detail & Related papers  (2024-04-08T08:04:44Z)
• SCINeRF: Neural Radiance Fields from a Snapshot Compressive Image [19.58894449169074]
  Snapshot Compressive Imaging (SCI) technique for recovering the underlying 3D scene representation from a single temporal compressed image. We formulate the physical imaging process of SCI as part of the training of neural radiance fields (NeRF) Our proposed approach surpasses the state-of-the-art methods in terms of image reconstruction and novel view image synthesis.
  arXiv  Detail & Related papers  (2024-03-29T07:14:14Z)
• GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation [91.01581867841894]
  We propose the GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts. Our GeoDiffusion is able to encode not only the bounding boxes but also extra geometric conditions such as camera views in self-driving scenes.
  arXiv  Detail & Related papers  (2023-06-07T17:17:58Z)
• Self-supervised Neural Networks for Spectral Snapshot Compressive Imaging [15.616674529295366]
  We consider using untrained neural networks to solve the reconstruction problem of snapshot compressive imaging (SCI) In this paper, inspired by the untrained neural networks such as deep image priors (DIP) and deep decoders, we develop a framework by integrating DIP into the plug-and-play regime, leading to a self-supervised network for spectral SCI reconstruction.
  arXiv  Detail & Related papers  (2021-08-28T14:17:38Z)
• 10-mega pixel snapshot compressive imaging with a hybrid coded aperture [48.95666098332693]
  High resolution images are widely used in our daily life, whereas high-speed video capture is challenging due to the low frame rate of cameras working at the high resolution mode. snapshot imaging (SCI) was proposed as a solution to the low throughput of existing imaging systems.
  arXiv  Detail & Related papers  (2021-06-30T01:09:24Z)
• Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Images [57.253447453301796]
  We introduce Deep Direct Volume Rendering (DeepDVR), a generalization of Direct Volume Rendering (DVR) that allows for the integration of deep neural networks into the DVR algorithm. We conceptualize the rendering in a latent color space, thus enabling the use of deep architectures to learn implicit mappings for feature extraction and classification. Our generalization serves to derive novel volume rendering architectures that can be trained end-to-end directly from examples in image space.
  arXiv  Detail & Related papers  (2021-06-09T23:03:00Z)
• Fed-Sim: Federated Simulation for Medical Imaging [131.56325440976207]
  We introduce a physics-driven generative approach that consists of two learnable neural modules. We show that our data synthesis framework improves the downstream segmentation performance on several datasets.
  arXiv  Detail & Related papers  (2020-09-01T19:17:46Z)
• Graph Signal Processing for Geometric Data and Beyond: Theory and Applications [55.81966207837108]
  Graph Signal Processing (GSP) enables processing signals that reside on irregular domains. GSP methodologies for geometric data in a unified manner by bridging the connections between geometric data and graphs. Recently developed Graph Neural Networks (GNNs) interpret the operation of these networks from the perspective of GSP.
  arXiv  Detail & Related papers  (2020-08-05T03:20:16Z)
• Various Total Variation for Snapshot Video Compressive Imaging [7.601695814245209]
  snapshot compressive imaging (SCI) was proposed to capture the high-dimensional (usually 3D) images using a 2D sensor (detector) Following this, reconstruction algorithms are employed to retrieve the high-dimensional data. This paper aims to answer the question of which TV penalty (anisotropic TV, isotropic TV and vectorized TV) works best for video SCI reconstruction?
  arXiv  Detail & Related papers  (2020-05-16T16:20:56Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)
• Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging [43.50482493611073]
  Snapshot imaging (SCI) aims to capture the high-dimensional (usually 3D) images using a 2D sensor (detector) in a single snapshot. Applying SCI to deep-scale (HD or UHD) videos in our daily lives is still challenging. This paper develops fast flexible algorithms for SCI based on the plug-play framework.
  arXiv  Detail & Related papers  (2020-03-30T17:41:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.