Snapshot Compressed Imaging Based Single-Measurement Computer Vision for Videos

• URL: http://arxiv.org/abs/2501.15122v1
• Date: Sat, 25 Jan 2025 08:20:30 GMT
• Title: Snapshot Compressed Imaging Based Single-Measurement Computer Vision for Videos
• Authors: Fengpu Pan, Jiangtao Wen, Yuxing Han,
• Abstract summary: Snapshot imaging (SCI) is a promising technique for capturing high-speed video at low bandwidth and low power. SCI also faces challenges in low-lighting photon-limited and low-signal-to-noise-ratio image conditions. We propose a novel Compressive Denoising Autoencoder (CompDAE) using the STFormer architecture as the backbone.
• Score: 12.322783570127756
• License:
• Abstract: Snapshot compressive imaging (SCI) is a promising technique for capturing high-speed video at low bandwidth and low power, typically by compressing multiple frames into a single measurement. However, similar to traditional CMOS image sensor based imaging systems, SCI also faces challenges in low-lighting photon-limited and low-signal-to-noise-ratio image conditions. In this paper, we propose a novel Compressive Denoising Autoencoder (CompDAE) using the STFormer architecture as the backbone, to explicitly model noise characteristics and provide computer vision functionalities such as edge detection and depth estimation directly from compressed sensing measurements, while accounting for realistic low-photon conditions. We evaluate the effectiveness of CompDAE across various datasets and demonstrated significant improvements in task performance compared to conventional RGB-based methods. In the case of ultra-low-lighting (APC $\leq$ 20) while conventional methods failed, the proposed algorithm can still maintain competitive performance.

Related papers

• Enhanced Confocal Laser Scanning Microscopy with Adaptive Physics Informed Deep Autoencoders [0.0]
  We present a physics-informed deep learning framework to address limitations in Confocal Laser Scanning Microscopy. The model reconstructs high fidelity images from heavily noisy inputs by using convolutional and transposed convolutional layers.
  arXiv  Detail & Related papers  (2025-01-24T18:32:34Z)
• Rethinking High-speed Image Reconstruction Framework with Spike Camera [48.627095354244204]
  Spike cameras generate continuous spike streams to capture high-speed scenes with lower bandwidth and higher dynamic range than traditional RGB cameras. We introduce a novel spike-to-image reconstruction framework SpikeCLIP that goes beyond traditional training paradigms. Our experiments on real-world low-light datasets demonstrate that SpikeCLIP significantly enhances texture details and the luminance balance of recovered images.
  arXiv  Detail & Related papers  (2025-01-08T13:00:17Z)
• Cross-Scan Mamba with Masked Training for Robust Spectral Imaging [51.557804095896174]
  We propose the Cross-Scanning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding. Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.
  arXiv  Detail & Related papers  (2024-08-01T15:14:10Z)
• Hybrid Training of Denoising Networks to Improve the Texture Acutance of Digital Cameras [3.400056739248712]
  We propose a mixed training procedure for image restoration neural networks, relying on both natural and synthetic images, that yields a strong improvement of this acutance metric without impairing fidelity terms. The feasibility of the approach is demonstrated both on the denoising of RGB images and the full development of RAW images, opening the path to a systematic improvement of the texture acutance of real imaging devices.
  arXiv  Detail & Related papers  (2024-02-20T10:47:06Z)
• Transferable Learned Image Compression-Resistant Adversarial Perturbations [66.46470251521947]
  Adversarial attacks can readily disrupt the image classification system, revealing the vulnerability of DNN-based recognition tasks. We introduce a new pipeline that targets image classification models that utilize learned image compressors as pre-processing modules.
  arXiv  Detail & Related papers  (2024-01-06T03:03:28Z)
• LWGNet: Learned Wirtinger Gradients for Fourier Ptychographic Phase Retrieval [14.588976801396576]
  We propose a hybrid model-driven residual network that combines the knowledge of the forward imaging system with a deep data-driven network. Unlike other conventional unrolling techniques, LWGNet uses fewer stages while performing at par or even better than existing traditional and deep learning techniques. This improvement in performance for low-bit depth and low-cost sensors has the potential to bring down the cost of FPM imaging setup significantly.
  arXiv  Detail & Related papers  (2022-08-08T17:22:54Z)
• Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks [35.532434169432776]
  We propose a lightweight two-stage image enhancement algorithm sequentially balancing illumination and noise removal. We also propose a Fourier spectrum-based adversarial framework (AFNet) for consistent image enhancement under varying illumination conditions. Based on quantitative and qualitative evaluations, we also examine the practicality and effects of image enhancement techniques on the performance of common perception tasks.
  arXiv  Detail & Related papers  (2022-04-04T18:48:51Z)
• Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder [73.48927855855219]
  We propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends. Experiments demonstrate that our approach successfully improves the rate-distortion performance over JPEG across various quality metrics.
  arXiv  Detail & Related papers  (2022-01-27T20:20:03Z)
• Burst Imaging for Light-Constrained Structure-From-Motion [4.125187280299246]
  We develop an image processing technique for aiding 3D reconstruction from images acquired in low light conditions. Our technique, based on burst photography, uses direct methods for image registration within bursts of short exposure time images. Our method is a significant step towards allowing robots to operate in low light conditions, with potential applications to robots operating in environments such as underground mines and night time operation.
  arXiv  Detail & Related papers  (2021-08-23T02:12:40Z)
• 10-mega pixel snapshot compressive imaging with a hybrid coded aperture [48.95666098332693]
  High resolution images are widely used in our daily life, whereas high-speed video capture is challenging due to the low frame rate of cameras working at the high resolution mode. snapshot imaging (SCI) was proposed as a solution to the low throughput of existing imaging systems.
  arXiv  Detail & Related papers  (2021-06-30T01:09:24Z)
• Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems [56.154190098338965]
  We show that our proposed time multiplexed coded aperture (TMCA) can be optimized end-to-end. TMCA induces better coded snapshots enabling superior reconstructions in two different applications: compressive light field imaging and hyperspectral imaging. This codification outperforms the state-of-the-art compressive imaging systems by more than 4dB in those applications.
  arXiv  Detail & Related papers  (2021-04-06T22:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.