Enhanced Neuromorphic Semantic Segmentation Latency through Stream Event

• URL: http://arxiv.org/abs/2502.18982v1
• Date: Wed, 26 Feb 2025 09:43:18 GMT
• Title: Enhanced Neuromorphic Semantic Segmentation Latency through Stream Event
• Authors: D. Hareb, J. Martinet, B. Miramond,
• Abstract summary: Achieving optimal semantic segmentation with frame-based vision sensors poses significant challenges for real-time systems like UAVs and self-driving cars.<n>We leverage event streams from event-based cameras-bio-inspired sensors that trigger events in response to changes in the scene.<n>We exploit this event information to solve the semantic segmentation task by employing a Spiking Neural Network (SNN), a bio-inspired computing paradigm known for its low energy consumption.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Achieving optimal semantic segmentation with frame-based vision sensors poses significant challenges for real-time systems like UAVs and self-driving cars, which require rapid and precise processing. Traditional frame-based methods often struggle to balance latency, accuracy, and energy efficiency. To address these challenges, we leverage event streams from event-based cameras-bio-inspired sensors that trigger events in response to changes in the scene. Specifically, we analyze the number of events triggered between successive frames, with a high number indicating significant changes and a low number indicating minimal changes. We exploit this event information to solve the semantic segmentation task by employing a Spiking Neural Network (SNN), a bio-inspired computing paradigm known for its low energy consumption. Our experiments on the DSEC dataset show that our approach significantly reduces latency with only a limited drop in accuracy. Additionally, by using SNNs, we achieve low power consumption, making our method suitable for energy-constrained real-time applications. To the best of our knowledge, our approach is the first to effectively balance reduced latency, minimal accuracy loss, and energy efficiency using events stream to enhance semantic segmentation in dynamic and resource-limited environments.

Related papers

• Spatiotemporal Attention Learning Framework for Event-Driven Object Recognition [1.0445957451908694]
  Event-based vision sensors capture local pixel-level intensity changes as a sparse event stream containing position, polarity, and information. This paper presents a novel learning framework for event-based object recognition, utilizing a VARGG network enhanced with Contemporalal Block Attention Module (CBAM) Our approach achieves comparable performance to state-of-the-art ResNet-based methods while reducing parameter count by 2.3% compared to the original VGG model.
  arXiv  Detail & Related papers  (2025-04-01T02:37:54Z)
• Event-based vision for egomotion estimation using precise event timing [0.6262316762195913]
  Egomotion estimation is crucial for applications such as autonomous navigation and robotics.<n>Traditional methods relying on inertial sensors are sensitive to external conditions.<n> Vision-based methods provide an efficient alternative by capturing data only when changes are perceived in the scene.
  arXiv  Detail & Related papers  (2025-01-20T15:41:33Z)
• CREST: An Efficient Conjointly-trained Spike-driven Framework for Event-based Object Detection Exploiting Spatiotemporal Dynamics [7.696109414724968]
  Spiking neural networks (SNNs) are promising for event-based object recognition and detection.<n>Existing SNN frameworks often fail to handle multi-scaletemporal features, leading to increased data redundancy and reduced accuracy.<n>We propose CREST, a novel conjointly-trained spike-driven framework to exploit event-based object detection.
  arXiv  Detail & Related papers  (2024-12-17T04:33:31Z)
• Spiking Neural Network as Adaptive Event Stream Slicer [10.279359105384334]
  Event-based cameras provide rich edge information, high dynamic range, and high temporal resolution. Many state-of-the-art event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information. SpikeSlicer is a novel-designed plug-and-play event processing method capable of splitting events stream adaptively.
  arXiv  Detail & Related papers  (2024-10-03T06:41:10Z)
• Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks [50.32980443749865]
  Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biologicalability. Current SNNs struggle to balance accuracy and latency in neuromorphic datasets. We propose Step-wise Distillation (HSD) method, tailored for neuromorphic datasets.
  arXiv  Detail & Related papers  (2024-09-19T06:52:34Z)
• Exploring Dynamic Transformer for Efficient Object Tracking [58.120191254379854]
  We propose DyTrack, a dynamic transformer framework for efficient tracking. DyTrack automatically learns to configure proper reasoning routes for various inputs, gaining better utilization of the available computational budget. Experiments on multiple benchmarks demonstrate that DyTrack achieves promising speed-precision trade-offs with only a single model.
  arXiv  Detail & Related papers  (2024-03-26T12:31:58Z)
• PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search [64.28335667655129]
  Multiple object tracking is a critical task in autonomous driving. As tracking accuracy improves, neural networks become increasingly complex, posing challenges for their practical application in real driving scenarios due to the high level of latency. In this paper, we explore the use of the neural architecture search (NAS) methods to search for efficient architectures for tracking, aiming for low real-time latency while maintaining relatively high accuracy.
  arXiv  Detail & Related papers  (2024-03-23T04:18:49Z)
• Low-power event-based face detection with asynchronous neuromorphic hardware [2.0774873363739985]
  We present the first instance of an on-chip spiking neural network for event-based face detection deployed on the SynSense Speck neuromorphic chip. We show how to reduce precision discrepancies between off-chip clock-driven simulation used for training and on-chip event-driven inference. We achieve an on-chip face detection mAP[0.5] of 0.6 while consuming only 20 mW.
  arXiv  Detail & Related papers  (2023-12-21T19:23:02Z)
• AEGNN: Asynchronous Event-based Graph Neural Networks [54.528926463775946]
  Event-based Graph Neural Networks generalize standard GNNs to process events as "evolving"-temporal graphs. AEGNNs are easily trained on synchronous inputs and can be converted to efficient, "asynchronous" networks at test time.
  arXiv  Detail & Related papers  (2022-03-31T16:21:12Z)
• Neuromorphic Camera Denoising using Graph Neural Network-driven Transformers [3.805262583092311]
  Neuromorphic vision is a bio-inspired technology that has triggered a paradigm shift in the computer-vision community. Neuromorphic cameras suffer from significant amounts of measurement noise. This noise deteriorates the performance of neuromorphic event-based perception and navigation algorithms.
  arXiv  Detail & Related papers  (2021-12-17T18:57:36Z)
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision [64.71260357476602]
  Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks. We propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection.
  arXiv  Detail & Related papers  (2021-12-06T23:45:58Z)
• Energy-Efficient Model Compression and Splitting for Collaborative Inference Over Time-Varying Channels [52.60092598312894]
  We propose a technique to reduce the total energy bill at the edge device by utilizing model compression and time-varying model split between the edge and remote nodes. Our proposed solution results in minimal energy consumption and $CO$ emission compared to the considered baselines.
  arXiv  Detail & Related papers  (2021-06-02T07:36:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.