Event Transformer. A sparse-aware solution for efficient event data processing

• URL: http://arxiv.org/abs/2204.03355v1
• Date: Thu, 7 Apr 2022 10:49:17 GMT
• Title: Event Transformer. A sparse-aware solution for efficient event data processing
• Authors: Alberto Sabater and Luis Montesano and Ana C. Murillo
• Abstract summary: Event Transformer (EvT) is a framework that effectively takes advantage of event-data properties to be highly efficient and accurate. EvT is evaluated on different event-based benchmarks for action and gesture recognition. Results show better or comparable accuracy to the state-of-the-art while requiring significantly less computation resources.
• Score: 9.669942356088377
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Event cameras are sensors of great interest for many applications that run in low-resource and challenging environments. They log sparse illumination changes with high temporal resolution and high dynamic range, while they present minimal power consumption. However, top-performing methods often ignore specific event-data properties, leading to the development of generic but computationally expensive algorithms. Efforts toward efficient solutions usually do not achieve top-accuracy results for complex tasks. This work proposes a novel framework, Event Transformer (EvT), that effectively takes advantage of event-data properties to be highly efficient and accurate. We introduce a new patch-based event representation and a compact transformer-like architecture to process it. EvT is evaluated on different event-based benchmarks for action and gesture recognition. Evaluation results show better or comparable accuracy to the state-of-the-art while requiring significantly less computation resources, which makes EvT able to work with minimal latency both on GPU and CPU.

Related papers

• Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba [11.400397931501338]
  Event cameras efficiently detect changes in ambient light with low latency and high dynamic range while consuming minimal power. Most current approach to processing event data often involves converting it into frame-based representations. Point Cloud is a popular representation for 3D processing and is better suited to match the sparse and asynchronous nature of the event camera. We propose EventMamba, an efficient and effective Point Cloud framework that achieves competitive results even compared to the state-of-the-art (SOTA) frame-based method.
  arXiv  Detail & Related papers  (2024-05-09T21:47:46Z)
• Scene Adaptive Sparse Transformer for Event-based Object Detection [40.04162039970849]
  We propose a Scene Adaptive Sparse Transformer (SAST) for event-based object detection. SAST enables window-token co-sparsification, significantly enhancing fault tolerance and reducing computational overhead. It outperforms all other dense and sparse networks in both performance and efficiency on two large-scale event-based object detection datasets.
  arXiv  Detail & Related papers  (2024-04-02T12:15:25Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• EventTransAct: A video transformer-based framework for Event-camera based action recognition [52.537021302246664]
  Event cameras offer new opportunities compared to standard action recognition in RGB videos. In this study, we employ a computationally efficient model, namely the video transformer network (VTN), which initially acquires spatial embeddings per event-frame. In order to better adopt the VTN for the sparse and fine-grained nature of event data, we design Event-Contrastive Loss ($mathcalL_EC$) and event-specific augmentations.
  arXiv  Detail & Related papers  (2023-08-25T23:51:07Z)
• FuzzyFlow: Leveraging Dataflow To Find and Squash Program Optimization Bugs [92.47146416628965]
  FuzzyFlow is a fault localization and test case extraction framework designed to test program optimizations. We leverage dataflow program representations to capture a fully reproducible system state and area-of-effect for optimizations. To reduce testing time, we design an algorithm for minimizing test inputs, trading off memory for recomputation.
  arXiv  Detail & Related papers  (2023-06-28T13:00:17Z)
• Energy-efficient Task Adaptation for NLP Edge Inference Leveraging Heterogeneous Memory Architectures [68.91874045918112]
  adapter-ALBERT is an efficient model optimization for maximal data reuse across different tasks. We demonstrate the advantage of mapping the model to a heterogeneous on-chip memory architecture by performing simulations on a validated NLP edge accelerator.
  arXiv  Detail & Related papers  (2023-03-25T14:40:59Z)
• Event Transformer+. A multi-purpose solution for efficient event data processing [13.648678472312374]
  Event cameras record sparse illumination changes with high temporal resolution and high dynamic range. Current methods often ignore specific event-data properties, leading to the development of generic but computationally expensive algorithms. We propose Event Transformer+, that improves our seminal work EvT with a refined patch-based event representation.
  arXiv  Detail & Related papers  (2022-11-22T12:28:37Z)
• RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer [63.25665813125223]
  We propose RTFormer, an efficient dual-resolution transformer for real-time semantic segmenation. It achieves better trade-off between performance and efficiency than CNN-based models. Experiments on mainstream benchmarks demonstrate the effectiveness of our proposed RTFormer.
  arXiv  Detail & Related papers  (2022-10-13T16:03:53Z)
• Ev-TTA: Test-Time Adaptation for Event-Based Object Recognition [7.814941658661939]
  Ev-TTA is a simple, effective test-time adaptation for event-based object recognition. Our formulation can be successfully applied regardless of input representations and extended into regression tasks.
  arXiv  Detail & Related papers  (2022-03-23T07:43:44Z)
• Time-Ordered Recent Event (TORE) Volumes for Event Cameras [21.419206807872797]
  Event cameras are an exciting, new sensor modality enabling high-speed imaging with extremely low-latency and wide dynamic range. Most machine learning architectures are not designed to directly handle sparse data, like that generated from event cameras. This paper details an event representation called Time-Ordered Recent Event (TORE) volumes. TORE volumes are designed to compactly store raw spike timing information with minimal information loss.
  arXiv  Detail & Related papers  (2021-03-10T15:03:38Z)
• Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation [53.850686395708905]
  Event-based cameras record an asynchronous stream of per-pixel brightness changes. In this paper, we focus on single-layer architectures for representation learning from event data. We show improvements of up to 9 % in the recognition accuracy compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-23T10:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.