Seeing the Unseen in Low-light Spike Streams

• URL: http://arxiv.org/abs/2509.23304v1
• Date: Sat, 27 Sep 2025 13:33:03 GMT
• Title: Seeing the Unseen in Low-light Spike Streams
• Authors: Liwen Hu, Yang Li, Mianzhi Liu, Yijia Guo, Shenghao Xie, Ziluo Ding, Tiejun Huang, Lei Ma,
• Abstract summary: Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks.<n>Due to unique data modality, spike streams require reconstruction methods to become perceptible to the human eye.<n>We propose Diff-SPK, the first diffusion-based reconstruction method for spike camera.
• Score: 31.57903575317722
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks. Unlike traditional cameras, spike camera continuously accumulates photons and fires asynchronous spike streams. Due to unique data modality, spike streams require reconstruction methods to become perceptible to the human eye. However, lots of methods struggle to handle spike streams in low-light high-speed scenarios due to severe noise and sparse information. In this work, we propose Diff-SPK, the first diffusion-based reconstruction method for spike camera. Diff-SPK effectively leverages generative priors to supplement texture information in low-light conditions. Specifically, it first employs an \textbf{E}nhanced \textbf{T}exture \textbf{f}rom Inter-spike \textbf{I}nterval (ETFI) to aggregate sparse information from low-light spike streams. Then, ETFI serves as a conditioning input for ControlNet to generate the high-speed scenes. To improve the quality of results, we introduce an ETFI-based feature fusion module during the generation process. Moreover, we establish the first bona fide benchmark for the low-light spike stream reconstruction task. It significantly surpasses existing reconstruction datasets in scale and provides quantitative illumination information. The performance on real low-light spike streams demonstrates the superiority of Diff-SPK.

Related papers

• SpikeStereoNet: A Brain-Inspired Framework for Stereo Depth Estimation from Spike Streams [43.43061247688823]
  Bio-inspired spike cameras emit asynchronous events at microsecond-level resolution, providing an alternative sensing modality.<n>Existing methods lack specialized stereo algorithms and benchmarks tailored to the spike data.<n>We propose SpikeStereoNet, a brain-inspired framework and the first to estimate stereo depth directly from raw spike streams.
  arXiv  Detail & Related papers  (2025-05-26T04:14:34Z)
• Rethinking High-speed Image Reconstruction Framework with Spike Camera [48.627095354244204]
  Spike cameras generate continuous spike streams to capture high-speed scenes with lower bandwidth and higher dynamic range than traditional RGB cameras.<n>We introduce a novel spike-to-image reconstruction framework SpikeCLIP that goes beyond traditional training paradigms.<n>Our experiments on real-world low-light datasets demonstrate that SpikeCLIP significantly enhances texture details and the luminance balance of recovered images.
  arXiv  Detail & Related papers  (2025-01-08T13:00:17Z)
• SwinSF: Image Reconstruction from Spatial-Temporal Spike Streams [2.609896297570564]
  We introduce Swin Spikeformer (SwinSF), a novel model for dynamic scene reconstruction from spike streams. SwinSF combines shifted window self-attention and proposed temporal spike attention, ensuring a comprehensive feature extraction. We build a new synthesized dataset for spike image reconstruction which matches the resolution of the latest spike camera.
  arXiv  Detail & Related papers  (2024-07-22T15:17:39Z)
• Learning to Robustly Reconstruct Low-light Dynamic Scenes from Spike Streams [28.258022350623023]
  As a neuromorphic sensor, spike camera can generate continuous binary spike streams to capture per-pixel light intensity. We propose a bidirectional recurrent-based reconstruction framework, including a Light-Robust Representation (LR-Rep) and a fusion module. We have developed a reconstruction benchmark for high-speed low-light scenes.
  arXiv  Detail & Related papers  (2024-01-19T03:01:07Z)
• LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models [54.93010869546011]
  We propose to leverage the pre-trained latent diffusion model to perform the neural ISP for enhancing extremely low-light images.<n>Specifically, to tailor the pre-trained latent diffusion model to operate on the RAW domain, we train a set of lightweight taming modules.<n>We observe different roles of UNet denoising and decoder reconstruction in the latent diffusion model, which inspires us to decompose the low-light image enhancement task into latent-space low-frequency content generation and decoding-phase high-frequency detail maintenance.
  arXiv  Detail & Related papers  (2023-12-02T04:31:51Z)
• Recurrent Spike-based Image Restoration under General Illumination [21.630646894529065]
  Spike camera is a new type of bio-inspired vision sensor that records light intensity in the form of a spike array with high temporal resolution (20,000 Hz) Existing spike-based approaches typically assume that the scenes are with sufficient light intensity, which is usually unavailable in many real-world scenarios such as rainy days or dusk scenes. We propose a Recurrent Spike-based Image Restoration (RSIR) network, which is the first work towards restoring clear images from spike arrays under general illumination.
  arXiv  Detail & Related papers  (2023-08-06T04:24:28Z)
• Spike Stream Denoising via Spike Camera Simulation [64.11994763727631]
  We propose a systematic noise model for spike camera based on its unique circuit. The first benchmark for spike stream denoising is proposed which includes clear (noisy) spike stream. Experiments show that DnSS has promising performance on the proposed benchmark.
  arXiv  Detail & Related papers  (2023-04-06T14:59:48Z)
• SCFlow: Optical Flow Estimation for Spiking Camera [50.770803466875364]
  Spiking camera has enormous potential in real applications, especially for motion estimation in high-speed scenes. Optical flow estimation has achieved remarkable success in image-based and event-based vision, but % existing methods cannot be directly applied in spike stream from spiking camera. This paper presents, SCFlow, a novel deep learning pipeline for optical flow estimation for spiking camera.
  arXiv  Detail & Related papers  (2021-10-08T06:16:45Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.