Quanta Burst Photography

• URL: http://arxiv.org/abs/2006.11840v1
• Date: Sun, 21 Jun 2020 16:20:29 GMT
• Title: Quanta Burst Photography
• Authors: Sizhuo Ma, Shantanu Gupta, Arin C. Ulku, Claudio Bruschini, Edoardo Charbon, Mohit Gupta
• Abstract summary: Single-photon avalanche diodes (SPADs) are an emerging sensor technology capable of detecting individual incident photons. We present quanta burst photography, a computational photography technique that leverages SPCs as passive imaging devices for photography in challenging conditions.
• Score: 15.722085082004934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Single-photon avalanche diodes (SPADs) are an emerging sensor technology capable of detecting individual incident photons, and capturing their time-of-arrival with high timing precision. While these sensors were limited to single-pixel or low-resolution devices in the past, recently, large (up to 1 MPixel) SPAD arrays have been developed. These single-photon cameras (SPCs) are capable of capturing high-speed sequences of binary single-photon images with no read noise. We present quanta burst photography, a computational photography technique that leverages SPCs as passive imaging devices for photography in challenging conditions, including ultra low-light and fast motion. Inspired by recent success of conventional burst photography, we design algorithms that align and merge binary sequences captured by SPCs into intensity images with minimal motion blur and artifacts, high signal-to-noise ratio (SNR), and high dynamic range. We theoretically analyze the SNR and dynamic range of quanta burst photography, and identify the imaging regimes where it provides significant benefits. We demonstrate, via a recently developed SPAD array, that the proposed method is able to generate high-quality images for scenes with challenging lighting, complex geometries, high dynamic range and moving objects. With the ongoing development of SPAD arrays, we envision quanta burst photography finding applications in both consumer and scientific photography.

Related papers

• bit2bit: 1-bit quanta video reconstruction via self-supervised photon prediction [57.199618102578576]
  We propose bit2bit, a new method for reconstructing high-quality image stacks at original resolution from sparse binary quantatemporal image data. Inspired by recent work on Poisson denoising, we developed an algorithm that creates a dense image sequence from sparse binary photon data. We present a novel dataset containing a wide range of real SPAD high-speed videos under various challenging imaging conditions.
  arXiv  Detail & Related papers  (2024-10-30T17:30:35Z)
• Photon Inhibition for Energy-Efficient Single-Photon Imaging [19.816230454712585]
  Single-photon cameras (SPCs) are emerging as sensors of choice for challenging imaging applications. Yet, single-photon sensitivity in SPADs comes at a cost -- each photon detection consumes more energy than that of a CMOS camera. We propose a computational-imaging approach called emphphoton inhibition to address this challenge.
  arXiv  Detail & Related papers  (2024-09-26T23:19:44Z)
• Radiance Fields from Photons [18.15183252935672]
  We introduce quanta radiance fields, a class of neural radiance fields that are trained at the granularity of individual photons using single-photon cameras (SPCs) We demonstrate, both via simulations and a prototype SPC hardware, high-fidelity reconstructions under high-speed motion, in low light, and for extreme dynamic range settings.
  arXiv  Detail & Related papers  (2024-07-12T16:06:51Z)
• Event Cameras Meet SPADs for High-Speed, Low-Bandwidth Imaging [25.13346470561497]
  Event cameras and single-photon avalanche diode (SPAD) sensors have emerged as promising alternatives to conventional cameras. We show that these properties are complementary, and can help achieve low-light, high-speed image reconstruction with low bandwidth requirements.
  arXiv  Detail & Related papers  (2024-04-17T16:06:29Z)
• Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor [58.305341034419136]
  We present the first dense SLAM system with a monocular camera and a light-weight ToF sensor. We propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor. Experiments demonstrate that our system well exploits the signals of light-weight ToF sensors and achieves competitive results.
  arXiv  Detail & Related papers  (2023-08-28T07:56:13Z)
• Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography [54.36608424943729]
  We show that in a ''long-burst'', forty-two 12-megapixel RAW frames captured in a two-second sequence, there is enough parallax information from natural hand tremor alone to recover high-quality scene depth. We devise a test-time optimization approach that fits a neural RGB-D representation to long-burst data and simultaneously estimates scene depth and camera motion.
  arXiv  Detail & Related papers  (2022-12-22T18:54:34Z)
• Simulating single-photon detector array sensors for depth imaging [2.497104612216142]
  Single-Photon Avalanche Detector (SPAD) arrays are a rapidly emerging technology. We establish a robust yet simple numerical procedure that establishes the fundamental limits to depth imaging with SPAD arrays.
  arXiv  Detail & Related papers  (2022-10-07T13:23:34Z)
• High Dynamic Range and Super-Resolution from Raw Image Bursts [52.341483902624006]
  This paper introduces the first approach to reconstruct high-resolution, high-dynamic range color images from raw photographic bursts captured by a handheld camera with exposure bracketing. The proposed algorithm is fast, with low memory requirements compared to state-of-the-art learning-based approaches to image restoration. Experiments demonstrate its excellent performance with super-resolution factors of up to $times 4$ on real photographs taken in the wild with hand-held cameras.
  arXiv  Detail & Related papers  (2022-07-29T13:31:28Z)
• Learning Spatially Varying Pixel Exposures for Motion Deblurring [49.07867902677453]
  We present a novel approach of leveraging spatially varying pixel exposures for motion deblurring. Our work illustrates the promising role that focal-plane sensor--processors can play in the future of computational imaging.
  arXiv  Detail & Related papers  (2022-04-14T23:41:49Z)
• Photon-Starved Scene Inference using Single Photon Cameras [14.121328731553868]
  We propose photon scale-space a collection of high-SNR images spanning a wide range of photons-per-pixel (PPP) levels. We develop training techniques that push images with different illumination levels closer to each other in feature representation space. Based on the proposed approach, we demonstrate, via simulations and real experiments with a SPAD camera, high-performance on various inference tasks.
  arXiv  Detail & Related papers  (2021-07-23T02:27:03Z)
• Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline [100.5353614588565]
  We propose to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.
  arXiv  Detail & Related papers  (2020-04-02T17:59:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.