Dynamic Low-light Imaging with Quanta Image Sensors
- URL: http://arxiv.org/abs/2007.08614v1
- Date: Thu, 16 Jul 2020 20:29:52 GMT
- Title: Dynamic Low-light Imaging with Quanta Image Sensors
- Authors: Yiheng Chi, Abhiram Gnanasambandam, Vladlen Koltun, Stanley H. Chan
- Abstract summary: We propose a solution using Quanta Image Sensors (QIS) and present a new image reconstruction algorithm.
We show that dynamic scenes can be reconstructed from a burst of frames at a photon level of 1 photon per pixel per frame.
- Score: 79.28256402267034
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Imaging in low light is difficult because the number of photons arriving at
the sensor is low. Imaging dynamic scenes in low-light environments is even
more difficult because as the scene moves, pixels in adjacent frames need to be
aligned before they can be denoised. Conventional CMOS image sensors (CIS) are
at a particular disadvantage in dynamic low-light settings because the exposure
cannot be too short lest the read noise overwhelms the signal. We propose a
solution using Quanta Image Sensors (QIS) and present a new image
reconstruction algorithm. QIS are single-photon image sensors with photon
counting capabilities. Studies over the past decade have confirmed the
effectiveness of QIS for low-light imaging but reconstruction algorithms for
dynamic scenes in low light remain an open problem. We fill the gap by
proposing a student-teacher training protocol that transfers knowledge from a
motion teacher and a denoising teacher to a student network. We show that
dynamic scenes can be reconstructed from a burst of frames at a photon level of
1 photon per pixel per frame. Experimental results confirm the advantages of
the proposed method compared to existing methods.
Related papers
- bit2bit: 1-bit quanta video reconstruction via self-supervised photon prediction [57.199618102578576]
We propose bit2bit, a new method for reconstructing high-quality image stacks at original resolution from sparse binary quantatemporal image data.
Inspired by recent work on Poisson denoising, we developed an algorithm that creates a dense image sequence from sparse binary photon data.
We present a novel dataset containing a wide range of real SPAD high-speed videos under various challenging imaging conditions.
arXiv Detail & Related papers (2024-10-30T17:30:35Z) - Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a
Light-Weight ToF Sensor [58.305341034419136]
We present the first dense SLAM system with a monocular camera and a light-weight ToF sensor.
We propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor.
Experiments demonstrate that our system well exploits the signals of light-weight ToF sensors and achieves competitive results.
arXiv Detail & Related papers (2023-08-28T07:56:13Z) - Make Explicit Calibration Implicit: Calibrate Denoiser Instead of the
Noise Model [83.9497193551511]
We introduce Lighting Every Darkness (LED), which is effective regardless of the digital gain or the camera sensor.
LED eliminates the need for explicit noise model calibration, instead utilizing an implicit fine-tuning process that allows quick deployment and requires minimal data.
LED also allows researchers to focus more on deep learning advancements while still utilizing sensor engineering benefits.
arXiv Detail & Related papers (2023-08-07T10:09:11Z) - Contrastive Learning for Low-light Raw Denoising [2.929093799984392]
We introduce a new denoising contrastive regularization (DCR) to exploit the information of noisy images and clean images.
In the feature space, DCR makes the denoised image closer to the clean image and far away from the noisy image.
In addition, we build a new feature embedding network called Wnet, which is more effective to extract high-frequency information.
arXiv Detail & Related papers (2023-05-05T08:13:53Z) - WildLight: In-the-wild Inverse Rendering with a Flashlight [77.31815397135381]
We propose a practical photometric solution for in-the-wild inverse rendering under unknown ambient lighting.
Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone.
We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques.
arXiv Detail & Related papers (2023-03-24T17:59:56Z) - Towards Robust Low Light Image Enhancement [6.85316573653194]
We study the problem of making brighter images from dark images found in the wild.
The images are dark because they are taken in dim environments. They suffer from color shifts caused by quantization and from sensor noise.
We use a supervised learning method, relying on a straightforward simulation of an imaging pipeline to generate usable dataset for training and testing.
arXiv Detail & Related papers (2022-05-17T20:14:18Z) - Enhancing Low-Light Images in Real World via Cross-Image Disentanglement [58.754943762945864]
We propose a new low-light image enhancement dataset consisting of misaligned training images with real-world corruptions.
Our model achieves state-of-the-art performances on both the newly proposed dataset and other popular low-light datasets.
arXiv Detail & Related papers (2022-01-10T03:12:52Z) - Photon-Starved Scene Inference using Single Photon Cameras [14.121328731553868]
We propose photon scale-space a collection of high-SNR images spanning a wide range of photons-per-pixel (PPP) levels.
We develop training techniques that push images with different illumination levels closer to each other in feature representation space.
Based on the proposed approach, we demonstrate, via simulations and real experiments with a SPAD camera, high-performance on various inference tasks.
arXiv Detail & Related papers (2021-07-23T02:27:03Z) - HDR Imaging with Quanta Image Sensors: Theoretical Limits and Optimal
Reconstruction [17.931673459050792]
We propose a new computational photography technique for HDR imaging.
We use the Quanta Image Sensor (QIS) to trade the spatial-temporal resolution with bit-depth.
We derive an optimal reconstruction algorithm for single-bit and multi-bit QIS.
arXiv Detail & Related papers (2020-11-06T22:08:03Z) - Image Classification in the Dark using Quanta Image Sensors [17.931673459050792]
We present a new low-light image classification solution using Quanta Image Sensors (QIS)
QIS are a new type of image sensors that possess photon counting ability without compromising on pixel size and spatial resolution.
We show that with student-teacher learning, we are able to achieve image classification at a photon level of one photon per pixel or lower.
arXiv Detail & Related papers (2020-06-03T03:39:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.