10-mega pixel snapshot compressive imaging with a hybrid coded aperture

• URL: http://arxiv.org/abs/2106.15765v1
• Date: Wed, 30 Jun 2021 01:09:24 GMT
• Title: 10-mega pixel snapshot compressive imaging with a hybrid coded aperture
• Authors: Zhihong Zhang, Chao Deng, Yang Liu, Xin Yuan, Jinli Suo, Qionghai Dai
• Abstract summary: High resolution images are widely used in our daily life, whereas high-speed video capture is challenging due to the low frame rate of cameras working at the high resolution mode. snapshot imaging (SCI) was proposed as a solution to the low throughput of existing imaging systems.
• Score: 48.95666098332693
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High resolution images are widely used in our daily life, whereas high-speed video capture is challenging due to the low frame rate of cameras working at the high resolution mode. Digging deeper, the main bottleneck lies in the low throughput of existing imaging systems. Towards this end, snapshot compressive imaging (SCI) was proposed as a promising solution to improve the throughput of imaging systems by compressive sampling and computational reconstruction. During acquisition, multiple high-speed images are encoded and collapsed to a single measurement. After this, algorithms are employed to retrieve the video frames from the coded snapshot. Recently developed Plug-and-Play (PnP) algorithms make it possible for SCI reconstruction in large-scale problems. However, the lack of high-resolution encoding systems still precludes SCI's wide application. In this paper, we build a novel hybrid coded aperture snapshot compressive imaging (HCA-SCI) system by incorporating a dynamic liquid crystal on silicon and a high-resolution lithography mask. We further implement a PnP reconstruction algorithm with cascaded denoisers for high quality reconstruction. Based on the proposed HCA-SCI system and algorithm, we achieve a 10-mega pixel SCI system to capture high-speed scenes, leading to a high throughput of 4.6G voxels per second. Both simulation and real data experiments verify the feasibility and performance of our proposed HCA-SCI scheme.

Related papers

• A Simple Low-bit Quantization Framework for Video Snapshot Compressive Imaging [15.351152482692383]
  Video Snapshot Compressive Imaging (SCI) aims to use a low-speed 2D camera to capture high-speed scene as snapshot compressed measurements. Deep learning-based algorithms have achieved impressive performance, yet with heavy computational workload. We propose a low-bit quantization framework (dubbed Q-SCI) for the end-to-end deep learning-based video SCI reconstruction methods.
  arXiv  Detail & Related papers  (2024-07-31T10:38:11Z)
• Deep Optics for Video Snapshot Compressive Imaging [10.830072985735175]
  Video snapshot imaging (SCI) aims to capture a sequence of video frames with only a single shot of a 2D detector. This paper presents a framework to jointly optimize masks and a reconstruction network. We believe this is a milestone for real-world video SCI.
  arXiv  Detail & Related papers  (2024-04-08T08:04:44Z)
• Sign-Coded Exposure Sensing for Noise-Robust High-Speed Imaging [16.58669052286989]
  We present a novel optical compression of high-speed frames employing pixel-level sign-coded exposure. Walsh functions ensure that the noise is not amplified during high-speed frame reconstruction. Our hardware prototype demonstrated the reconstruction of 4kHz frames of a moving scene lit by ambient light only.
  arXiv  Detail & Related papers  (2023-05-05T01:03:37Z)
• Rolling Shutter Inversion: Bring Rolling Shutter Images to High Framerate Global Shutter Video [111.08121952640766]
  This paper presents a novel deep-learning based solution to the RS temporal super-resolution problem. By leveraging the multi-view geometry relationship of the RS imaging process, our framework successfully achieves high framerate GS generation. Our method can produce high-quality GS image sequences with rich details, outperforming the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-10-06T16:47:12Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems [56.154190098338965]
  We show that our proposed time multiplexed coded aperture (TMCA) can be optimized end-to-end. TMCA induces better coded snapshots enabling superior reconstructions in two different applications: compressive light field imaging and hyperspectral imaging. This codification outperforms the state-of-the-art compressive imaging systems by more than 4dB in those applications.
  arXiv  Detail & Related papers  (2021-04-06T22:42:34Z)
• Memory-Efficient Network for Large-scale Video Compressive Sensing [21.040260603729227]
  Video snapshot imaging (SCI) captures a sequence of video frames in a single shot using a 2D detector. In this paper, we develop a memory-efficient network for large-scale video SCI based on multi-group reversible 3D convolutional neural networks.
  arXiv  Detail & Related papers  (2021-03-04T15:14:58Z)
• Deep Unfolded Recovery of Sub-Nyquist Sampled Ultrasound Image [94.42139459221784]
  We propose a reconstruction method from sub-Nyquist samples in the time and spatial domain, that is based on unfolding the ISTA algorithm. Our method allows reducing the number of array elements, sampling rate, and computational time while ensuring high quality imaging performance.
  arXiv  Detail & Related papers  (2021-03-01T19:19:38Z)
• Plug-and-Play Algorithms for Video Snapshot Compressive Imaging [41.818167109996885]
  We consider the reconstruction problem of snapshot video imaging (SCI) using a low-speed 2D sensor (detector) The underlying principle SCI is to modulate frames with different masks and then encoded frames are integrated into a snapshot on the sensor. Applying SCI to largescale problems (HD or UHD videos) in our daily life is still challenging one bottlenecks lies in the reconstruction algorithm.
  arXiv  Detail & Related papers  (2021-01-13T00:51:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.