Photon Inhibition for Energy-Efficient Single-Photon Imaging

• URL: http://arxiv.org/abs/2409.18337v1
• Date: Thu, 26 Sep 2024 23:19:44 GMT
• Title: Photon Inhibition for Energy-Efficient Single-Photon Imaging
• Authors: Lucas J. Koerner, Shantanu Gupta, Atul Ingle, Mohit Gupta,
• Abstract summary: Single-photon cameras (SPCs) are emerging as sensors of choice for challenging imaging applications. Yet, single-photon sensitivity in SPADs comes at a cost -- each photon detection consumes more energy than that of a CMOS camera. We propose a computational-imaging approach called emphphoton inhibition to address this challenge.
• Score: 19.816230454712585
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Single-photon cameras (SPCs) are emerging as sensors of choice for various challenging imaging applications. One class of SPCs based on the single-photon avalanche diode (SPAD) detects individual photons using an avalanche process; the raw photon data can then be processed to extract scene information under extremely low light, high dynamic range, and rapid motion. Yet, single-photon sensitivity in SPADs comes at a cost -- each photon detection consumes more energy than that of a CMOS camera. This avalanche power significantly limits sensor resolution and could restrict widespread adoption of SPAD-based SPCs. We propose a computational-imaging approach called \emph{photon inhibition} to address this challenge. Photon inhibition strategically allocates detections in space and time based on downstream inference task goals and resource constraints. We develop lightweight, on-sensor computational inhibition policies that use past photon data to disable SPAD pixels in real-time, to select the most informative future photons. As case studies, we design policies tailored for image reconstruction and edge detection, and demonstrate, both via simulations and real SPC captured data, considerable reduction in photon detections (over 90\% of photons) while maintaining task performance metrics. Our work raises the question of ``which photons should be detected?'', and paves the way for future energy-efficient single-photon imaging.

Related papers

• bit2bit: 1-bit quanta video reconstruction via self-supervised photon prediction [57.199618102578576]
  We propose bit2bit, a new method for reconstructing high-quality image stacks at original resolution from sparse binary quantatemporal image data. Inspired by recent work on Poisson denoising, we developed an algorithm that creates a dense image sequence from sparse binary photon data. We present a novel dataset containing a wide range of real SPAD high-speed videos under various challenging imaging conditions.
  arXiv  Detail & Related papers  (2024-10-30T17:30:35Z)
• Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor [58.305341034419136]
  We present the first dense SLAM system with a monocular camera and a light-weight ToF sensor. We propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor. Experiments demonstrate that our system well exploits the signals of light-weight ToF sensors and achieves competitive results.
  arXiv  Detail & Related papers  (2023-08-28T07:56:13Z)
• Number-State Reconstruction with a Single Single-Photon Avalanche Detector [1.5833270109954136]
  Single-photon avalanche detectors (SPADs) are crucial sensors of light for many fields and applications. We present a methodology for performing photon number-state reconstruction with only one SPAD.
  arXiv  Detail & Related papers  (2023-08-25T18:00:35Z)
• High-dimensional quantum correlation measurements with an adaptively gated hybrid single-photon camera [58.720142291102135]
  We propose an adaptively-gated hybrid intensified camera (HIC) that combines a high spatial resolution sensor and a high temporal resolution detector. With a spatial resolution of nearly 9 megapixels and nanosecond temporal resolution, this system allows for the realization of previously infeasible quantum optics experiments.
  arXiv  Detail & Related papers  (2023-05-25T16:59:27Z)
• On-chip quantum information processing with distinguishable photons [55.41644538483948]
  Multi-photon interference is at the heart of photonic quantum technologies. Here, we experimentally demonstrate that detection can be implemented with a temporal resolution sufficient to interfere photons detuned on the scales necessary for cavity-based integrated photon sources. We show how time-resolved detection of non-ideal photons can be used to improve the fidelity of an entangling operation and to mitigate the reduction of computational complexity in boson sampling experiments.
  arXiv  Detail & Related papers  (2022-10-14T18:16:49Z)
• A photosensor employing data-driven binning for ultrafast image recognition [0.0]
  Pixel binning is a technique widely used in optical image acquisition and spectroscopy. Here, we push the concept of binning to its limit by combining a large fraction of the sensor elements into a single superpixel. For a given pattern recognition task, its optimal shape is determined from training data using a machine learning algorithm.
  arXiv  Detail & Related papers  (2021-11-20T15:38:39Z)
• Photon-Starved Scene Inference using Single Photon Cameras [14.121328731553868]
  We propose photon scale-space a collection of high-SNR images spanning a wide range of photons-per-pixel (PPP) levels. We develop training techniques that push images with different illumination levels closer to each other in feature representation space. Based on the proposed approach, we demonstrate, via simulations and real experiments with a SPAD camera, high-performance on various inference tasks.
  arXiv  Detail & Related papers  (2021-07-23T02:27:03Z)
• Passive Inter-Photon Imaging [18.739224941453983]
  Digital camera pixels measure image intensities by converting incident light energy into an analog electrical current, and then digitizing it into a fixed-width binary representation. This direct measurement method suffers from limited dynamic range and poor performance under extreme illumination. We propose a novel intensity cue based on measuring inter-photon timing, defined as the time delay between detection of successive photons.
  arXiv  Detail & Related papers  (2021-03-31T18:44:52Z)
• Dynamic Low-light Imaging with Quanta Image Sensors [79.28256402267034]
  We propose a solution using Quanta Image Sensors (QIS) and present a new image reconstruction algorithm. We show that dynamic scenes can be reconstructed from a burst of frames at a photon level of 1 photon per pixel per frame.
  arXiv  Detail & Related papers  (2020-07-16T20:29:52Z)
• Quanta Burst Photography [15.722085082004934]
  Single-photon avalanche diodes (SPADs) are an emerging sensor technology capable of detecting individual incident photons. We present quanta burst photography, a computational photography technique that leverages SPCs as passive imaging devices for photography in challenging conditions.
  arXiv  Detail & Related papers  (2020-06-21T16:20:29Z)
• Deep Photon Mapping [59.41146655216394]
  In this paper, we develop the first deep learning-based method for particle-based rendering. We train a novel deep neural network to predict a kernel function to aggregate photon contributions at shading points. Our network encodes individual photons into per-photon features, aggregates them in the neighborhood of a shading point, and infers a kernel function from the per-photon and photon local context features.
  arXiv  Detail & Related papers  (2020-04-25T06:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.