Plug-and-Play Algorithms for Video Snapshot Compressive Imaging
- URL: http://arxiv.org/abs/2101.04822v1
- Date: Wed, 13 Jan 2021 00:51:49 GMT
- Title: Plug-and-Play Algorithms for Video Snapshot Compressive Imaging
- Authors: Xin Yuan, Yang Liu, Jinli Suo, Fr\'edo Durand, Qionghai Dai
- Abstract summary: We consider the reconstruction problem of snapshot video imaging (SCI) using a low-speed 2D sensor (detector)
The underlying principle SCI is to modulate frames with different masks and then encoded frames are integrated into a snapshot on the sensor.
Applying SCI to largescale problems (HD or UHD videos) in our daily life is still challenging one bottlenecks lies in the reconstruction algorithm.
- Score: 41.818167109996885
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We consider the reconstruction problem of video snapshot compressive imaging
(SCI), which captures high-speed videos using a low-speed 2D sensor (detector).
The underlying principle of SCI is to modulate sequential high-speed frames
with different masks and then these encoded frames are integrated into a
snapshot on the sensor and thus the sensor can be of low-speed. On one hand,
video SCI enjoys the advantages of low-bandwidth, low-power and low-cost. On
the other hand, applying SCI to large-scale problems (HD or UHD videos) in our
daily life is still challenging and one of the bottlenecks lies in the
reconstruction algorithm. Exiting algorithms are either too slow (iterative
optimization algorithms) or not flexible to the encoding process (deep learning
based end-to-end networks). In this paper, we develop fast and flexible
algorithms for SCI based on the plug-and-play (PnP) framework. In addition to
the PnP-ADMM method, we further propose the PnP-GAP (generalized alternating
projection) algorithm with a lower computational workload. We first employ the
image deep denoising priors to show that PnP can recover a UHD color video with
30 frames from a snapshot measurement. Since videos have strong temporal
correlation, by employing the video deep denoising priors, we achieve a
significant improvement in the results. Furthermore, we extend the proposed PnP
algorithms to the color SCI system using mosaic sensors, where each pixel only
captures the red, green or blue channels. A joint reconstruction and
demosaicing paradigm is developed for flexible and high quality reconstruction
of color video SCI systems. Extensive results on both simulation and real
datasets verify the superiority of our proposed algorithm.
Related papers
- Deep Optics for Video Snapshot Compressive Imaging [10.830072985735175]
Video snapshot imaging (SCI) aims to capture a sequence of video frames with only a single shot of a 2D detector.
This paper presents a framework to jointly optimize masks and a reconstruction network.
We believe this is a milestone for real-world video SCI.
arXiv Detail & Related papers (2024-04-08T08:04:44Z) - VNVC: A Versatile Neural Video Coding Framework for Efficient
Human-Machine Vision [59.632286735304156]
It is more efficient to enhance/analyze the coded representations directly without decoding them into pixels.
We propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis.
arXiv Detail & Related papers (2023-06-19T03:04:57Z) - EfficientSCI: Densely Connected Network with Space-time Factorization
for Large-scale Video Snapshot Compressive Imaging [6.8372546605486555]
We show that an UHD color video with high compression ratio can be reconstructed from a snapshot 2D measurement using a single end-to-end deep learning model with PSNR above 32 dB.
Our method significantly outperforms all previous SOTA algorithms with better real-time performance.
arXiv Detail & Related papers (2023-05-17T07:28:46Z) - Small Lesion Segmentation in Brain MRIs with Subpixel Embedding [105.1223735549524]
We present a method to segment MRI scans of the human brain into ischemic stroke lesion and normal tissues.
We propose a neural network architecture in the form of a standard encoder-decoder where predictions are guided by a spatial expansion embedding network.
arXiv Detail & Related papers (2021-09-18T00:21:17Z) - 10-mega pixel snapshot compressive imaging with a hybrid coded aperture [48.95666098332693]
High resolution images are widely used in our daily life, whereas high-speed video capture is challenging due to the low frame rate of cameras working at the high resolution mode.
snapshot imaging (SCI) was proposed as a solution to the low throughput of existing imaging systems.
arXiv Detail & Related papers (2021-06-30T01:09:24Z) - Memory-Efficient Network for Large-scale Video Compressive Sensing [21.040260603729227]
Video snapshot imaging (SCI) captures a sequence of video frames in a single shot using a 2D detector.
In this paper, we develop a memory-efficient network for large-scale video SCI based on multi-group reversible 3D convolutional neural networks.
arXiv Detail & Related papers (2021-03-04T15:14:58Z) - Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames.
We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs.
We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
arXiv Detail & Related papers (2020-08-20T20:01:59Z) - Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging [43.50482493611073]
Snapshot imaging (SCI) aims to capture the high-dimensional (usually 3D) images using a 2D sensor (detector) in a single snapshot.
Applying SCI to deep-scale (HD or UHD) videos in our daily lives is still challenging.
This paper develops fast flexible algorithms for SCI based on the plug-play framework.
arXiv Detail & Related papers (2020-03-30T17:41:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.