Flow-Based Visual Stream Compression for Event Cameras

• URL: http://arxiv.org/abs/2403.08086v1
• Date: Tue, 12 Mar 2024 21:36:41 GMT
• Title: Flow-Based Visual Stream Compression for Event Cameras
• Abstract summary: We introduce a flow-based method for the real-time compression of event streams as they are generated. This method leverages real-time optical flow estimates to predict future events without needing to transmit them. We demonstrate that the proposed prediction algorithm is capable of performing real-time, low-latency event prediction.
• Score: 3.533874233403883
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the use of neuromorphic, event-based vision sensors expands, the need for compression of their output streams has increased. While their operational principle ensures event streams are spatially sparse, the high temporal resolution of the sensors can result in high data rates from the sensor depending on scene dynamics. For systems operating in communication-bandwidth-constrained and power-constrained environments, it is essential to compress these streams before transmitting them to a remote receiver. Therefore, we introduce a flow-based method for the real-time asynchronous compression of event streams as they are generated. This method leverages real-time optical flow estimates to predict future events without needing to transmit them, therefore, drastically reducing the amount of data transmitted. The flow-based compression introduced is evaluated using a variety of methods including spatiotemporal distance between event streams. The introduced method itself is shown to achieve an average compression ratio of 2.81 on a variety of event-camera datasets with the evaluation configuration used. That compression is achieved with a median temporal error of 0.48 ms and an average spatiotemporal event-stream distance of 3.07. When combined with LZMA compression for non-real-time applications, our method can achieve state-of-the-art average compression ratios ranging from 10.45 to 17.24. Additionally, we demonstrate that the proposed prediction algorithm is capable of performing real-time, low-latency event prediction.

Related papers

• VoRTeC: Taming Foundation Flow for One-step Real time Video Compression [38.79321243531853]
  $mathttVoRTeC$ is a Video Compression framework built upon a foundational flow model (Wan2.1)<n>Our method reduces bit consumption by 58% compared to prior diffusion-based approaches.<n>$mathttVoRTeC$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.
  arXiv  Detail & Related papers  (2026-09-02T08:39:23Z)
• Lossy Event Compression: From Event Stream Distortion to Task Performance [4.026541089436386]
  Lossy compression is essential for practical deployment of event cameras.<n>Existing event stream distortion metrics fail to reliably predict compression-induced degradation at the task level.<n>This paper introduces two fundamentally different event compression pipelines.
  arXiv  Detail & Related papers  (2026-08-28T15:12:56Z)
• Event-Based Motion Estimation via Oriented Distance Fields [55.23416502666468]
  Event-based motion estimation is central to tasks that demand high temporal resolution and robustness to fast motion.<n>We propose Oriented Distance Field Motion Estimation (ODF Motion Estimation), which replaces this optimization with a single averaging step over a precomputed field of event distance vectors.<n>On public and self-collected datasets, ODF motion estimation reaches sub-pixel accuracy at the lowest latency among compared methods.
  arXiv  Detail & Related papers  (2026-08-25T08:27:42Z)
• Event Signal Filtering via Probability Flux Estimation [58.31652473933809]
  Events offer a novel paradigm for capturing scene dynamics via asynchronous sensing, but their inherent randomness often leads to degraded signal quality. Event signal filtering is thus essential for enhancing fidelity by reducing this internal randomness and ensuring consistent outputs across diverse acquisition conditions. This paper introduces a generative, online filtering framework called Event Density Flow Filter (EDFilter) Experiments validate EDFilter's performance across tasks like event filtering, super-resolution, and direct event-based blob tracking.
  arXiv  Detail & Related papers  (2025-04-10T07:03:08Z)
• Neural-based Video Compression on Solar Dynamics Observatory Images [8.73521037463594]
  NASA's Solar Dynamics Observatory (SDO) mission collects extensive data to monitor the Sun's daily activity. Data compression plays a crucial role in addressing the challenges posed by limited telemetry rates. This paper introduces a neural video compression technique that achieves a high compression ratio for the SDO's image data collection.
  arXiv  Detail & Related papers  (2024-07-12T21:24:25Z)
• MemFlow: Optical Flow Estimation and Prediction with Memory [54.22820729477756]
  We present MemFlow, a real-time method for optical flow estimation and prediction with memory. Our method enables memory read-out and update modules for aggregating historical motion information in real-time. Our approach seamlessly extends to the future prediction of optical flow based on past observations.
  arXiv  Detail & Related papers  (2024-04-07T04:56:58Z)
• StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation [29.30999290150683]
  We introduce StreamDiffusion, a real-time diffusion pipeline for interactive image generation. Existing diffusion models are adept at creating images from text or image prompts, yet they often fall short in real-time interaction. We present a novel approach that transforms the original sequential denoising into the denoising process.
  arXiv  Detail & Related papers  (2023-12-19T18:18:33Z)
• Dynamic Frame Interpolation in Wavelet Domain [57.25341639095404]
  Video frame is an important low-level computation vision task, which can increase frame rate for more fluent visual experience. Existing methods have achieved great success by employing advanced motion models and synthesis networks. WaveletVFI can reduce computation up to 40% while maintaining similar accuracy, making it perform more efficiently against other state-of-the-arts.
  arXiv  Detail & Related papers  (2023-09-07T06:41:15Z)
• StreamingFlow: Streaming Occupancy Forecasting with Asynchronous Multi-modal Data Streams via Neural Ordinary Differential Equation [15.441175735210791]
  StreamingFlow is a novel BEV occupancy predictor that ingests asynchronous multi-sensor data streams for fusion. It learns derivatives of BEV features over temporal horizons, updates the implicit sensor's BEV features as part of the fusion process, and propagates BEV states to the desired future time point. It significantly outperforms previous vision-based, LiDAR-based methods, and shows superior performance compared to state-of-the-art fusion-based methods.
  arXiv  Detail & Related papers  (2023-02-19T14:38:01Z)
• Fast Event-based Optical Flow Estimation by Triplet Matching [13.298845944779108]
  Event cameras offer advantages over traditional cameras (low latency, high dynamic range, low power, etc.) Optical flow estimation methods that work on packets of events trade off speed for accuracy. We propose a novel optical flow estimation scheme based on triplet matching.
  arXiv  Detail & Related papers  (2022-12-23T09:12:16Z)
• Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis [40.249030338644225]
  Video-to-Video synthesis (Vid2Vid) has achieved remarkable results in generating a photo-realistic video from a sequence of semantic maps. Fast-Vid2Vid achieves around real-time performance as 20 FPS and saves around 8x computational cost on a single V100 GPU.
  arXiv  Detail & Related papers  (2022-07-11T17:57:57Z)
• Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction [50.361427832256524]
  We propose a coarse-to-fine (C2F) deep video compression framework for better motion compensation. Our C2F framework can achieve better motion compensation results without significantly increasing bit costs.
  arXiv  Detail & Related papers  (2022-06-15T11:38:53Z)
• SCFlow: Optical Flow Estimation for Spiking Camera [50.770803466875364]
  Spiking camera has enormous potential in real applications, especially for motion estimation in high-speed scenes. Optical flow estimation has achieved remarkable success in image-based and event-based vision, but % existing methods cannot be directly applied in spike stream from spiking camera. This paper presents, SCFlow, a novel deep learning pipeline for optical flow estimation for spiking camera.
  arXiv  Detail & Related papers  (2021-10-08T06:16:45Z)
• TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
  We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches. We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
  arXiv  Detail & Related papers  (2021-06-14T10:33:47Z)
• Energy-Efficient Model Compression and Splitting for Collaborative Inference Over Time-Varying Channels [52.60092598312894]
  We propose a technique to reduce the total energy bill at the edge device by utilizing model compression and time-varying model split between the edge and remote nodes. Our proposed solution results in minimal energy consumption and $CO$ emission compared to the considered baselines.
  arXiv  Detail & Related papers  (2021-06-02T07:36:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.