A Simple Low-bit Quantization Framework for Video Snapshot Compressive Imaging

• URL: http://arxiv.org/abs/2407.21517v1
• Date: Wed, 31 Jul 2024 10:38:11 GMT
• Title: A Simple Low-bit Quantization Framework for Video Snapshot Compressive Imaging
• Authors: Miao Cao, Lishun Wang, Huan Wang, Xin Yuan,
• Abstract summary: Video Snapshot Compressive Imaging (SCI) aims to use a low-speed 2D camera to capture high-speed scene as snapshot compressed measurements. Deep learning-based algorithms have achieved impressive performance, yet with heavy computational workload. We propose a low-bit quantization framework (dubbed Q-SCI) for the end-to-end deep learning-based video SCI reconstruction methods.
• Score: 15.351152482692383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video Snapshot Compressive Imaging (SCI) aims to use a low-speed 2D camera to capture high-speed scene as snapshot compressed measurements, followed by a reconstruction algorithm to reconstruct the high-speed video frames. State-of-the-art (SOTA) deep learning-based algorithms have achieved impressive performance, yet with heavy computational workload. Network quantization is a promising way to reduce computational cost. However, a direct low-bit quantization will bring large performance drop. To address this challenge, in this paper, we propose a simple low-bit quantization framework (dubbed Q-SCI) for the end-to-end deep learning-based video SCI reconstruction methods which usually consist of a feature extraction, feature enhancement, and video reconstruction module. Specifically, we first design a high-quality feature extraction module and a precise video reconstruction module to extract and propagate high-quality features in the low-bit quantized model. In addition, to alleviate the information distortion of the Transformer branch in the quantized feature enhancement module, we introduce a shift operation on the query and key distributions to further bridge the performance gap. Comprehensive experimental results manifest that our Q-SCI framework can achieve superior performance, e.g., 4-bit quantized EfficientSCI-S derived by our Q-SCI framework can theoretically accelerate the real-valued EfficientSCI-S by 7.8X with only 2.3% performance gap on the simulation testing datasets. Code is available at https://github.com/mcao92/QuantizedSCI.

Related papers

• PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution [87.89013794655207]
  Diffusion-based image super-resolution (SR) models have shown superior performance at the cost of multiple denoising steps. We propose a novel post-training quantization approach with adaptive scale in one-step diffusion (OSD) image SR, PassionSR. Our PassionSR achieves significant advantages over recent leading low-bit quantization methods for image SR.
  arXiv  Detail & Related papers  (2024-11-26T04:49:42Z)
• Compression-Realized Deep Structural Network for Video Quality Enhancement [78.13020206633524]
  This paper focuses on the task of quality enhancement for compressed videos. Most of the existing methods lack a structured design to optimally leverage the priors within compression codecs. A new paradigm is urgently needed for a more conscious'' process of quality enhancement.
  arXiv  Detail & Related papers  (2024-05-10T09:18:17Z)
• Hierarchical Frequency-based Upsampling and Refining for Compressed Video Quality Enhancement [14.653248860008981]
  We propose a hierarchical frequency-based upsampling and refining neural network (HFUR) for compressed video quality enhancement. ImpFreqUp exploits DCT-domain prior derived through implicit DCT transform, and accurately reconstructs the DCT-domain loss via a coarse-to-fine transfer. HIR is introduced to facilitate cross-collaboration and information compensation between the scales, thus further refine the feature maps and promote the visual quality of the final output.
  arXiv  Detail & Related papers  (2024-03-18T08:13:26Z)
• VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision [59.632286735304156]
  It is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. We propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis.
  arXiv  Detail & Related papers  (2023-06-19T03:04:57Z)
• Exploring Effective Mask Sampling Modeling for Neural Image Compression [171.35596121939238]
  Most existing neural image compression methods rely on side information from hyperprior or context models to eliminate spatial redundancy. Inspired by the mask sampling modeling in recent self-supervised learning methods for natural language processing and high-level vision, we propose a novel pretraining strategy for neural image compression. Our method achieves competitive performance with lower computational complexity compared to state-of-the-art image compression methods.
  arXiv  Detail & Related papers  (2023-06-09T06:50:20Z)
• EfficientSCI: Densely Connected Network with Space-time Factorization for Large-scale Video Snapshot Compressive Imaging [6.8372546605486555]
  We show that an UHD color video with high compression ratio can be reconstructed from a snapshot 2D measurement using a single end-to-end deep learning model with PSNR above 32 dB. Our method significantly outperforms all previous SOTA algorithms with better real-time performance.
  arXiv  Detail & Related papers  (2023-05-17T07:28:46Z)
• 10-mega pixel snapshot compressive imaging with a hybrid coded aperture [48.95666098332693]
  High resolution images are widely used in our daily life, whereas high-speed video capture is challenging due to the low frame rate of cameras working at the high resolution mode. snapshot imaging (SCI) was proposed as a solution to the low throughput of existing imaging systems.
  arXiv  Detail & Related papers  (2021-06-30T01:09:24Z)
• Memory-Efficient Network for Large-scale Video Compressive Sensing [21.040260603729227]
  Video snapshot imaging (SCI) captures a sequence of video frames in a single shot using a 2D detector. In this paper, we develop a memory-efficient network for large-scale video SCI based on multi-group reversible 3D convolutional neural networks.
  arXiv  Detail & Related papers  (2021-03-04T15:14:58Z)
• Plug-and-Play Algorithms for Video Snapshot Compressive Imaging [41.818167109996885]
  We consider the reconstruction problem of snapshot video imaging (SCI) using a low-speed 2D sensor (detector) The underlying principle SCI is to modulate frames with different masks and then encoded frames are integrated into a snapshot on the sensor. Applying SCI to largescale problems (HD or UHD videos) in our daily life is still challenging one bottlenecks lies in the reconstruction algorithm.
  arXiv  Detail & Related papers  (2021-01-13T00:51:49Z)
• Modeling Lost Information in Lossy Image Compression [72.69327382643549]
  Lossy image compression is one of the most commonly used operators for digital images. We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
  arXiv  Detail & Related papers  (2020-06-22T04:04:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.