HDR Imaging with Quanta Image Sensors: Theoretical Limits and Optimal Reconstruction

• URL: http://arxiv.org/abs/2011.03614v2
• Date: Wed, 2 Dec 2020 20:28:52 GMT
• Title: HDR Imaging with Quanta Image Sensors: Theoretical Limits and Optimal Reconstruction
• Authors: Abhiram Gnanasambandam and Stanley H. Chan
• Abstract summary: We propose a new computational photography technique for HDR imaging. We use the Quanta Image Sensor (QIS) to trade the spatial-temporal resolution with bit-depth. We derive an optimal reconstruction algorithm for single-bit and multi-bit QIS.
• Score: 17.931673459050792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High dynamic range (HDR) imaging is one of the biggest achievements in modern photography. Traditional solutions to HDR imaging are designed for and applied to CMOS image sensors (CIS). However, the mainstream one-micron CIS cameras today generally have a high read noise and low frame-rate. These, in turn, limit the acquisition speed and quality, making the cameras slow in the HDR mode. In this paper, we propose a new computational photography technique for HDR imaging. Recognizing the limitations of CIS, we use the Quanta Image Sensor (QIS) to trade the spatial-temporal resolution with bit-depth. QIS is a single-photon image sensor that has comparable pixel pitch to CIS but substantially lower dark current and read noise. We provide a complete theoretical characterization of the sensor in the context of HDR imaging, by proving the fundamental limits in the dynamic range that QIS can offer and the trade-offs with noise and speed. In addition, we derive an optimal reconstruction algorithm for single-bit and multi-bit QIS. Our algorithm is theoretically optimal for \emph{all} linear reconstruction schemes based on exposure bracketing. Experimental results confirm the validity of the theory and algorithm, based on synthetic and real QIS data.

Related papers

• bit2bit: 1-bit quanta video reconstruction via self-supervised photon prediction [57.199618102578576]
  We propose bit2bit, a new method for reconstructing high-quality image stacks at original resolution from sparse binary quantatemporal image data. Inspired by recent work on Poisson denoising, we developed an algorithm that creates a dense image sequence from sparse binary photon data. We present a novel dataset containing a wide range of real SPAD high-speed videos under various challenging imaging conditions.
  arXiv  Detail & Related papers  (2024-10-30T17:30:35Z)
• Event-based Asynchronous HDR Imaging by Temporal Incident Light Modulation [54.64335350932855]
  We propose a Pixel-Asynchronous HDR imaging system, based on key insights into the challenges in HDR imaging. Our proposed Asyn system integrates the Dynamic Vision Sensors (DVS) with a set of LCD panels. The LCD panels modulate the irradiance incident upon the DVS by altering their transparency, thereby triggering the pixel-independent event streams.
  arXiv  Detail & Related papers  (2024-03-14T13:45:09Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• RBSRICNN: Raw Burst Super-Resolution through Iterative Convolutional Neural Network [23.451063587138393]
  We propose a Raw Burst Super-Resolution Iterative Convolutional Neural Network (RBSRICNN) The proposed network produces the final output by an iterative refinement of the intermediate SR estimates. We demonstrate the effectiveness of our proposed approach in quantitative and qualitative experiments.
  arXiv  Detail & Related papers  (2021-10-25T19:01:28Z)
• Thermal Image Processing via Physics-Inspired Deep Networks [21.094006629684376]
  DeepIR combines physically accurate sensor modeling with deep network-based image representation. DeepIR requires neither training data nor periodic ground-truth calibration with a known black body target. Simulated and real data experiments demonstrate that DeepIR can perform high-quality non-uniformity correction with as few as three images.
  arXiv  Detail & Related papers  (2021-08-18T04:57:48Z)
• HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization [39.82945546614887]
  We propose a novel learning-based approach using a spatially dynamic encoder-decoder network, HDRUNet, to learn an end-to-end mapping for single image HDR reconstruction. Our method achieves the state-of-the-art performance in quantitative comparisons and visual quality.
  arXiv  Detail & Related papers  (2021-05-27T12:12:34Z)
• Beyond Visual Attractiveness: Physically Plausible Single Image HDR Reconstruction for Spherical Panoramas [60.24132321381606]
  We introduce the physical illuminance constraints to our single-shot HDR reconstruction framework. Our method can generate HDRs which are not only visually appealing but also physically plausible.
  arXiv  Detail & Related papers  (2021-03-24T01:51:19Z)
• Dynamic Low-light Imaging with Quanta Image Sensors [79.28256402267034]
  We propose a solution using Quanta Image Sensors (QIS) and present a new image reconstruction algorithm. We show that dynamic scenes can be reconstructed from a burst of frames at a photon level of 1 photon per pixel per frame.
  arXiv  Detail & Related papers  (2020-07-16T20:29:52Z)
• Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline [100.5353614588565]
  We propose to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.
  arXiv  Detail & Related papers  (2020-04-02T17:59:04Z)
• Adaptive Dithering Using Curved Markov-Gaussian Noise in the Quantized Domain for Mapping SDR to HDR Image [2.913398015606848]
  High Dynamic Range (SDR) imaging is gaining increased attention due to its realistic content, for not only regular displays but also smartphones. We present a technique for noise generation that operates on pixels of a quantized image. We vary the magnitude and structure of the noise pattern adaptively based on the luma of the quantized pixel and the slope of the inverse-tone mapping function.
  arXiv  Detail & Related papers  (2020-01-20T05:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.