v2e: From Video Frames to Realistic DVS Events

• URL: http://arxiv.org/abs/2006.07722v2
• Date: Mon, 19 Apr 2021 08:30:15 GMT
• Title: v2e: From Video Frames to Realistic DVS Events
• Abstract summary: This paper proposes the v2e toolbox that generates realistic synthetic DVS events from intensity frames. Realistic DVS events are useful in training networks for uncontrolled lighting conditions.
• Score: 16.562442770255032
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To help meet the increasing need for dynamic vision sensor (DVS) event camera data, this paper proposes the v2e toolbox that generates realistic synthetic DVS events from intensity frames. It also clarifies incorrect claims about DVS motion blur and latency characteristics in recent literature. Unlike other toolboxes, v2e includes pixel-level Gaussian event threshold mismatch, finite intensity-dependent bandwidth, and intensity-dependent noise. Realistic DVS events are useful in training networks for uncontrolled lighting conditions. The use of v2e synthetic events is demonstrated in two experiments. The first experiment is object recognition with N-Caltech 101 dataset. Results show that pretraining on various v2e lighting conditions improves generalization when transferred on real DVS data for a ResNet model. The second experiment shows that for night driving, a car detector trained with v2e events shows an average accuracy improvement of 40% compared to the YOLOv3 trained on intensity frames.

Related papers

• Pedestrian Crossing Intent Classification From Event-Based Vision Using Convolutional Spiking Neural Networks With Temporal Augmentation [0.0]
  We present an end-to-end pipeline that converts real-world driving footage into synthetic vision sensor (DVS) event streams.<n>We also train a novel convolutional spiking neural network (Conv-SNN) with clip-consistent DVS augmentation to classify pedestrian crossing intent as binary.<n>The trained model achieves 95.83% accuracy and F1 = 0.9695 on the JAAD DVS test set, 97.79% accuracy and F1 = 0.9478 on normal CARLA DVS, and 94.78% accuracy and F1 = 0.8369 on adverse-weather CARLA DVS.
  arXiv  Detail & Related papers  (2026-09-11T02:34:24Z)
• Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation [53.47090840868696]
  Video-to-audio (V2A) generation extends image-to-audio generation (I2A)<n>We introduce TD-V2A, which leverages temporal differences (TD) as the key representation that distinguishes V2A from I2A.<n>We show that effectively exploiting TD through our proposed framework significantly improves end-to-end V2A generation quality.
  arXiv  Detail & Related papers  (2026-08-05T14:28:05Z)
• DVS-PedX: Synthetic-and-Real Event-Based Pedestrian Dataset [0.0]
  Event cameras like Dynamic Vision Sensors (DVS) report micro-timed brightness changes instead of full frames.<n>DVS-PedX is a neuromorphic dataset designed for pedestrian detection and crossing-intention analysis in normal and adverse weather conditions.
  arXiv  Detail & Related papers  (2025-09-04T11:30:29Z)
• Event-Driven Dynamic Scene Depth Completion [50.01494043834177]
  EventDC is the first event-driven depth completion framework.<n>It consists of two key components: Event-Modulated Alignment (EMA) and Local Depth Filtering (LDF)
  arXiv  Detail & Related papers  (2025-05-19T16:02:37Z)
• Event2Vec: Processing Neuromorphic Events directly by Representations in Vector Space [11.383354549570436]
  Neuromorphic event cameras possess superior temporal resolution, power efficiency, and dynamic range compared to traditional cameras.<n> Existing solutions to this incompatibility often sacrifice temporal resolution, require extensive pre-processing, and do not fully leverage GPU acceleration.<n>We introduce event2vec, a novel representation that allows neural networks to process events directly.
  arXiv  Detail & Related papers  (2025-04-21T18:21:18Z)
• Restoring Real-World Degraded Events Improves Deblurring Quality [1.841128448495454]
  We model the degradation of events and propose RDNet to improve the quality of image deblurring. To better assess the deblurring performance of different methods on real-world degraded events, we present a new real-world dataset named DavisMCR.
  arXiv  Detail & Related papers  (2024-07-30T02:29:59Z)
• LaSe-E2V: Towards Language-guided Semantic-Aware Event-to-Video Reconstruction [8.163356555241322]
  We propose a novel framework, called LaSe-E2V, that can achieve semantic-aware high-quality E2V reconstruction. We first propose an Event-guided Spatiotemporal Attention (ESA) module to condition the event data to the denoising pipeline effectively. We then introduce an event-aware mask loss to ensure temporal coherence and a noise strategy to enhance spatial consistency.
  arXiv  Detail & Related papers  (2024-07-08T01:40:32Z)
• Characterization of dim light response in DVS pixel: Discontinuity of event triggering time [7.623462728752422]
  We analyze the typical Dynamic Vision Sensors (DVS) circuit, and find that there exists discontinuity of event triggering time. We point out that the discontinuity depends exclusively on the changing speed of light intensity.
  arXiv  Detail & Related papers  (2024-04-27T03:55:53Z)
• Temporal-Mapping Photography for Event Cameras [5.344756442054121]
  Event cameras, or Dynamic Vision Sensors (DVS), capture brightness changes as a continuous stream of "events" Converting sparse events to dense intensity frames faithfully has long been an ill-posed problem. In this paper, for the first time, we realize events to dense intensity image conversion using a stationary event camera in static scenes.
  arXiv  Detail & Related papers  (2024-03-11T05:29:46Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields [80.94515892378053]
  EvDNeRF is a pipeline for generating event data and training an event-based dynamic NeRF. NeRFs offer geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions.
  arXiv  Detail & Related papers  (2023-10-03T21:08:41Z)
• V2CE: Video to Continuous Events Simulator [1.1009908861287052]
  We present a novel method for video-to-events stream conversion from multiple perspectives, considering the specific characteristics of Dynamic Vision Sensor (DVS) A series of carefully designed timestamp losses helps enhance the quality of generated event voxels significantly. We also propose a novel local dynamic-aware inference strategy to accurately recover event timestamps from event voxels in a continuous fashion.
  arXiv  Detail & Related papers  (2023-09-16T06:06:53Z)
• Dual Memory Aggregation Network for Event-Based Object Detection with Learnable Representation [79.02808071245634]
  Event-based cameras are bio-inspired sensors that capture brightness change of every pixel in an asynchronous manner. Event streams are divided into grids in the x-y-t coordinates for both positive and negative polarity, producing a set of pillars as 3D tensor representation. Long memory is encoded in the hidden state of adaptive convLSTMs while short memory is modeled by computing spatial-temporal correlation between event pillars.
  arXiv  Detail & Related papers  (2023-03-17T12:12:41Z)
• EventNeRF: Neural Radiance Fields from a Single Colour Event Camera [81.19234142730326]
  This paper proposes the first approach for 3D-consistent, dense and novel view synthesis using just a single colour event stream as input. At its core is a neural radiance field trained entirely in a self-supervised manner from events while preserving the original resolution of the colour event channels. We evaluate our method qualitatively and numerically on several challenging synthetic and real scenes and show that it produces significantly denser and more visually appealing renderings.
  arXiv  Detail & Related papers  (2022-06-23T17:59:53Z)
• VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows [93.54888104118822]
  We propose a large-scale Visible-Event benchmark (termed VisEvent) due to the lack of a realistic and scaled dataset for this task. Our dataset consists of 820 video pairs captured under low illumination, high speed, and background clutter scenarios. Based on VisEvent, we transform the event flows into event images and construct more than 30 baseline methods.
  arXiv  Detail & Related papers  (2021-08-11T03:55:12Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.