Neuromorphic Imaging and Classification with Graph Learning

• URL: http://arxiv.org/abs/2309.15627v2
• Date: Thu, 21 Mar 2024 15:17:10 GMT
• Title: Neuromorphic Imaging and Classification with Graph Learning
• Authors: Pei Zhang, Chutian Wang, Edmund Y. Lam,
• Abstract summary: Bio-inspired neuromorphic cameras asynchronously record pixel brightness changes and generate sparse event streams. Due to the multidimensional address-event structure, most existing vision algorithms cannot properly handle asynchronous event streams. We propose a new graph representation of the event data and couple it with a Graph Transformer to perform accurate neuromorphic classification.
• Score: 11.882239213276392
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Bio-inspired neuromorphic cameras asynchronously record pixel brightness changes and generate sparse event streams. They can capture dynamic scenes with little motion blur and more details in extreme illumination conditions. Due to the multidimensional address-event structure, most existing vision algorithms cannot properly handle asynchronous event streams. While several event representations and processing methods have been developed to address such an issue, they are typically driven by a large number of events, leading to substantial overheads in runtime and memory. In this paper, we propose a new graph representation of the event data and couple it with a Graph Transformer to perform accurate neuromorphic classification. Extensive experiments show that our approach leads to better results and excels at the challenging realistic situations where only a small number of events and limited computational resources are available, paving the way for neuromorphic applications embedded into mobile facilities.

Related papers

• Spatio-temporal Transformers for Action Unit Classification with Event Cameras [28.98336123799572]
  We present FACEMORPHIC, a temporally synchronized multimodal face dataset composed of RGB videos and event streams. We show how temporal synchronization can allow effective neuromorphic face analysis without the need to manually annotate videos.
  arXiv  Detail & Related papers  (2024-10-29T11:23:09Z)
• EvDNeRF: Reconstructing Event Data with Dynamic Neural Radiance Fields [80.94515892378053]
  EvDNeRF is a pipeline for generating event data and training an event-based dynamic NeRF. NeRFs offer geometric-based learnable rendering, but prior work with events has only considered reconstruction of static scenes. We show that by training on varied batch sizes of events, we can improve test-time predictions of events at fine time resolutions.
  arXiv  Detail & Related papers  (2023-10-03T21:08:41Z)
• Deformable Neural Radiance Fields using RGB and Event Cameras [65.40527279809474]
  We develop a novel method to model the deformable neural radiance fields using RGB and event cameras. The proposed method uses the asynchronous stream of events and sparse RGB frames. Experiments conducted on both realistically rendered graphics and real-world datasets demonstrate a significant benefit of the proposed method.
  arXiv  Detail & Related papers  (2023-09-15T14:19:36Z)
• Graph-based Asynchronous Event Processing for Rapid Object Recognition [59.112755601918074]
  Event cameras capture asynchronous events stream in which each event encodes pixel location, trigger time, and the polarity of the brightness changes. We introduce a novel graph-based framework for event cameras, namely SlideGCN. Our approach can efficiently process data event-by-event, unlock the low latency nature of events data while still maintaining the graph's structure internally.
  arXiv  Detail & Related papers  (2023-08-28T08:59:57Z)
• EventNeRF: Neural Radiance Fields from a Single Colour Event Camera [81.19234142730326]
  This paper proposes the first approach for 3D-consistent, dense and novel view synthesis using just a single colour event stream as input. At its core is a neural radiance field trained entirely in a self-supervised manner from events while preserving the original resolution of the colour event channels. We evaluate our method qualitatively and numerically on several challenging synthetic and real scenes and show that it produces significantly denser and more visually appealing renderings.
  arXiv  Detail & Related papers  (2022-06-23T17:59:53Z)
• Event Transformer [43.193463048148374]
  Event camera's low power consumption and ability to capture microsecond brightness make it attractive for various computer vision tasks. Existing event representation methods typically convert events into frames, voxel grids, or spikes for deep neural networks (DNNs) This work introduces a novel token-based event representation, where each event is considered a fundamental processing unit termed an event-token.
  arXiv  Detail & Related papers  (2022-04-11T15:05:06Z)
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts [51.17064599766138]
  We have developed a method to identify independently objects acquired with an event-based camera. The method performs on par or better than the state of the art without having to predetermine the number of expected moving objects.
  arXiv  Detail & Related papers  (2020-12-16T04:06:02Z)
• Matching Neuromorphic Events and Color Images via Adversarial Learning [49.447580124957966]
  We propose the Event-Based Image Retrieval (EBIR) problem to exploit the cross-modal matching task. We address the EBIR problem by proposing neuromorphic Events-Color image Feature Learning (ECFL) We also contribute to the community N-UKbench and EC180 dataset to promote the development of EBIR problem.
  arXiv  Detail & Related papers  (2020-03-02T02:48:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.