Event Transformer+. A multi-purpose solution for efficient event data processing

• URL: http://arxiv.org/abs/2211.12222v2
• Date: Sun, 3 Sep 2023 10:09:33 GMT
• Title: Event Transformer+. A multi-purpose solution for efficient event data processing
• Authors: Alberto Sabater, Luis Montesano, Ana C. Murillo
• Abstract summary: Event cameras record sparse illumination changes with high temporal resolution and high dynamic range. Current methods often ignore specific event-data properties, leading to the development of generic but computationally expensive algorithms. We propose Event Transformer+, that improves our seminal work EvT with a refined patch-based event representation.
• Score: 13.648678472312374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Event cameras record sparse illumination changes with high temporal resolution and high dynamic range. Thanks to their sparse recording and low consumption, they are increasingly used in applications such as AR/VR and autonomous driving. Current topperforming methods often ignore specific event-data properties, leading to the development of generic but computationally expensive algorithms, while event-aware methods do not perform as well. We propose Event Transformer+, that improves our seminal work EvT with a refined patch-based event representation and a more robust backbone to achieve more accurate results, while still benefiting from event-data sparsity to increase its efficiency. Additionally, we show how our system can work with different data modalities and propose specific output heads, for event-stream classification (i.e. action recognition) and per-pixel predictions (dense depth estimation). Evaluation results show better performance to the state-of-the-art while requiring minimal computation resources, both on GPU and CPU.

Related papers

• Event-to-Video Conversion for Overhead Object Detection [7.744259147081667]
  Event cameras complicate downstream image processing, especially for complex tasks such as object detection. We show that there is a significant gap in performance between dense event representations and corresponding RGB frames. We apply event-to-video conversion models that convert event streams into gray-scale video to close this gap.
  arXiv  Detail & Related papers  (2024-02-09T22:07:39Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields [80.94515892378053]
  EvDNeRF is a pipeline for generating event data and training an event-based dynamic NeRF. NeRFs offer geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions.
  arXiv  Detail & Related papers  (2023-10-03T21:08:41Z)
• Graph-based Asynchronous Event Processing for Rapid Object Recognition [59.112755601918074]
  Event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes. We introduce a novel graph-based framework for event cameras, namely SlideGCN. Our approach can efficiently process data event-by-event, unlock the low latency nature of events data while still maintaining the graph's structure internally.
  arXiv  Detail & Related papers  (2023-08-28T08:59:57Z)
• EventMix: An Efficient Augmentation Strategy for Event-Based Data [4.8416725611508244]
  Event cameras can provide high dynamic range and low-energy event stream data. The scale is smaller and more difficult to obtain than traditional frame-based data. This paper proposes an efficient data augmentation strategy for event stream data: EventMix.
  arXiv  Detail & Related papers  (2022-05-24T13:07:33Z)
• Event Transformer. A sparse-aware solution for efficient event data processing [9.669942356088377]
  Event Transformer (EvT) is a framework that effectively takes advantage of event-data properties to be highly efficient and accurate. EvT is evaluated on different event-based benchmarks for action and gesture recognition. Results show better or comparable accuracy to the state-of-the-art while requiring significantly less computation resources.
  arXiv  Detail & Related papers  (2022-04-07T10:49:17Z)
• Asynchronous Optimisation for Event-based Visual Odometry [53.59879499700895]
  Event cameras open up new possibilities for robotic perception due to their low latency and high dynamic range. We focus on event-based visual odometry (VO) We propose an asynchronous structure-from-motion optimisation back-end.
  arXiv  Detail & Related papers  (2022-03-02T11:28:47Z)
• Time-Ordered Recent Event (TORE) Volumes for Event Cameras [21.419206807872797]
  Event cameras are an exciting, new sensor modality enabling high-speed imaging with extremely low-latency and wide dynamic range. Most machine learning architectures are not designed to directly handle sparse data, like that generated from event cameras. This paper details an event representation called Time-Ordered Recent Event (TORE) volumes. TORE volumes are designed to compactly store raw spike timing information with minimal information loss.
  arXiv  Detail & Related papers  (2021-03-10T15:03:38Z)
• Dynamic Resource-aware Corner Detection for Bio-inspired Vision Sensors [0.9988653233188148]
  We present an algorithm to detect asynchronous corners from a stream of events in real-time on embedded systems. The proposed algorithm is capable of selecting the best corner candidate among neighbors and achieves an average execution time savings of 59 %.
  arXiv  Detail & Related papers  (2020-10-29T12:01:33Z)
• Learning Monocular Dense Depth from Events [53.078665310545745]
  Event cameras produce brightness changes in the form of a stream of asynchronous events instead of intensity frames. Recent learning-based approaches have been applied to event-based data, such as monocular depth prediction. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods.
  arXiv  Detail & Related papers  (2020-10-16T12:36:23Z)
• Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation [53.850686395708905]
  Event-based cameras record an asynchronous stream of per-pixel brightness changes. In this paper, we focus on single-layer architectures for representation learning from event data. We show improvements of up to 9 % in the recognition accuracy compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-23T10:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.