Optical Flow from Photons

• URL: http://arxiv.org/abs/2608.00499v1
• Date: Sat, 01 Aug 2026 07:45:19 GMT
• Title: Optical Flow from Photons
• Abstract summary: Single-photon avalanche diode (SPAD) cameras offer single-photon sensitivity and extremely fine temporal sampling.<n>However, individual slices in these high FPS binary photon streams are too sparse for dense correspondence.<n>We propose QuantaFlow, the first method for dense optical flow directly from SPAD streams.
• Score: 45.38756990937711
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical flow remains challenging in high-speed and low-light scenes, where the limited frame rate and sensitivity of conventional cameras lead to motion blur and underexposure. Single-photon avalanche diode (SPAD) cameras offer single-photon sensitivity and extremely fine temporal sampling. However, individual slices in these high FPS binary photon streams are too sparse for dense correspondence. Temporal aggregation can provide the spatial cues required by optical flow, but accumulating photons at fixed coordinates blurs moving structures. Motion-aware aggregation can reduce this blur, yet it depends on the flow being estimated. To address this dependency, we propose QuantaFlow, the first method for dense optical flow directly from SPAD streams. Instead of constructing a fixed input representation, QuantaFlow embeds SPAD representation construction into iterative flow refinement. At each iteration, the current flow coarsely aligns the slices within the source and target sub-streams. A photon-flux transformation then constructs multi-scale representations containing intensity and structural cues, while adaptive multi-scale fusion balances photon noise and residual motion blur at each pixel. The fused representations drive a feature-warping flow update, and the refined flow guides representation construction in the next iteration. We further construct a synthetic dataset for SPAD optical-flow training and evaluation. Experiments on the synthetic dataset and real-world SPAD data demonstrate the effectiveness and generalization of QuantaFlow.

Related papers

• OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolation [55.676358801492114]
  We propose OCAI, a method that supports robust frame ambiguities by generating intermediate video frames alongside optical flows in between. Our evaluations demonstrate superior quality and enhanced optical flow accuracy on established benchmarks such as Sintel and KITTI.
  arXiv  Detail & Related papers  (2024-03-26T20:23:48Z)
• Towards Anytime Optical Flow Estimation with Event Cameras [37.892840680923584]
  Event cameras respond to changes in log-brightness at the millisecond level, making them ideal for optical flow estimation.<n>Existing datasets from event cameras provide only low frame rate ground truth for optical flow.<n>We propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow.
  arXiv  Detail & Related papers  (2023-07-11T06:15:12Z)
• Deep Dynamic Scene Deblurring from Optical Flow [53.625999196063574]
  Deblurring can provide visually more pleasant pictures and make photography more convenient. It is difficult to model the non-uniform blur mathematically. We develop a convolutional neural network (CNN) to restore the sharp images from the deblurred features.
  arXiv  Detail & Related papers  (2023-01-18T06:37:21Z)
• Learning Dense and Continuous Optical Flow from an Event Camera [28.77846425802558]
  Event cameras such as DAVIS can simultaneously output high temporal resolution events and low frame-rate intensity images. Most of the existing optical flow estimation methods are based on two consecutive image frames and can only estimate discrete flow at a fixed time interval. We propose a novel deep learning-based dense and continuous optical flow estimation framework from a single image with event streams.
  arXiv  Detail & Related papers  (2022-11-16T17:53:18Z)
• RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos [28.995525297929348]
  RealFlow is a framework that can create large-scale optical flow datasets directly from unlabeled realistic videos. We first estimate optical flow between a pair of video frames, and then synthesize a new image from this pair based on the predicted flow. Our approach achieves state-of-the-art performance on two standard benchmarks compared with both supervised and unsupervised optical flow methods.
  arXiv  Detail & Related papers  (2022-07-22T13:33:03Z)
• SCFlow: Optical Flow Estimation for Spiking Camera [50.770803466875364]
  Spiking camera has enormous potential in real applications, especially for motion estimation in high-speed scenes. Optical flow estimation has achieved remarkable success in image-based and event-based vision, but % existing methods cannot be directly applied in spike stream from spiking camera. This paper presents, SCFlow, a novel deep learning pipeline for optical flow estimation for spiking camera.
  arXiv  Detail & Related papers  (2021-10-08T06:16:45Z)
• Dense Optical Flow from Event Cameras [55.79329250951028]
  We propose to incorporate feature correlation and sequential processing into dense optical flow estimation from event cameras. Our proposed approach computes dense optical flow and reduces the end-point error by 23% on MVSEC.
  arXiv  Detail & Related papers  (2021-08-24T07:39:08Z)
• OmniFlow: Human Omnidirectional Optical Flow [0.0]
  Our paper presents OmniFlow: a new synthetic omnidirectional human optical flow dataset. Based on a rendering engine we create a naturalistic 3D indoor environment with textured rooms, characters, actions, objects, illumination and motion blur. The simulation has as output rendered images of household activities and the corresponding forward and backward optical flow.
  arXiv  Detail & Related papers  (2021-04-16T08:25:20Z)
• Learning optical flow from still images [53.295332513139925]
  We introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. We virtually move the camera in the reconstructed environment with known motion vectors and rotation angles. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data.
  arXiv  Detail & Related papers  (2021-04-08T17:59:58Z)
• Fusion-FlowNet: Energy-Efficient Optical Flow Estimation using Sensor Fusion and Deep Fused Spiking-Analog Network Architectures [7.565038387344594]
  We present a sensor fusion framework for energy-efficient optical flow estimation using both frame- and event-based sensors. Our network is end-to-end trained using unsupervised learning to avoid expensive video annotations.
  arXiv  Detail & Related papers  (2021-03-19T02:03:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.