Efficient Object Detection in Autonomous Driving using Spiking Neural Networks: Performance, Energy Consumption Analysis, and Insights into Open-set Object Discovery

• URL: http://arxiv.org/abs/2312.07466v1
• Date: Tue, 12 Dec 2023 17:47:13 GMT
• Title: Efficient Object Detection in Autonomous Driving using Spiking Neural Networks: Performance, Energy Consumption Analysis, and Insights into Open-set Object Discovery
• Authors: Aitor Martinez Seras, Javier Del Ser, Pablo Garcia-Bringas
• Abstract summary: A well-balanced trade-off between performance and energy consumption is crucial for the sustainability of autonomous vehicles. We show that well-performing and efficient models can be realized by virtue of Spiking Neural Networks.
• Score: 8.255197802529118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Besides performance, efficiency is a key design driver of technologies supporting vehicular perception. Indeed, a well-balanced trade-off between performance and energy consumption is crucial for the sustainability of autonomous vehicles. In this context, the diversity of real-world contexts in which autonomous vehicles can operate motivates the need for empowering perception models with the capability to detect, characterize and identify newly appearing objects by themselves. In this manuscript we elaborate on this threefold conundrum (performance, efficiency and open-world learning) for object detection modeling tasks over image data collected from vehicular scenarios. Specifically, we show that well-performing and efficient models can be realized by virtue of Spiking Neural Networks (SNNs), reaching competitive levels of detection performance when compared to their non-spiking counterparts at dramatic energy consumption savings (up to 85%) and a slightly improved robustness against image noise. Our experiments herein offered also expose qualitatively the complexity of detecting new objects based on the preliminary results of a simple approach to discriminate potential object proposals in the captured image.

Related papers

• AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
  We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios. We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
  arXiv  Detail & Related papers  (2024-03-26T04:27:56Z)
• Towards In-Vehicle Multi-Task Facial Attribute Recognition: Investigating Synthetic Data and Vision Foundation Models [8.54530542456452]
  We investigate the utility of synthetic datasets for training complex multi-task models that recognize facial attributes of passengers of a vehicle. Our study unveils counter-intuitive findings, notably the superior performance of ResNet over ViTs in our specific multi-task context.
  arXiv  Detail & Related papers  (2024-03-10T04:17:54Z)
• 3D Object Visibility Prediction in Autonomous Driving [6.802572869909114]
  We present a novel attribute and its corresponding algorithm: 3D object visibility. Our proposal of this attribute and its computational strategy aims to expand the capabilities for downstream tasks.
  arXiv  Detail & Related papers  (2024-03-06T13:07:42Z)
• Elastic Interaction Energy-Informed Real-Time Traffic Scene Perception [8.429178814528617]
  A topology-aware energy loss function-based network training strategy named EIEGSeg is proposed. EIEGSeg is designed for multi-class segmentation on real-time traffic scene perception. Our results demonstrate that EIEGSeg consistently improves the performance, especially on real-time, lightweight networks.
  arXiv  Detail & Related papers  (2023-10-02T01:30:42Z)
• Exploring Model Transferability through the Lens of Potential Energy [78.60851825944212]
  Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models. Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels. We present an insightful physics-inspired approach named PED to address these challenges.
  arXiv  Detail & Related papers  (2023-08-29T07:15:57Z)
• Understanding Self-attention Mechanism via Dynamical System Perspective [58.024376086269015]
  Self-attention mechanism (SAM) is widely used in various fields of artificial intelligence. We show that intrinsic stiffness phenomenon (SP) in the high-precision solution of ordinary differential equations (ODEs) also widely exists in high-performance neural networks (NN) We show that the SAM is also a stiffness-aware step size adaptor that can enhance the model's representational ability to measure intrinsic SP.
  arXiv  Detail & Related papers  (2023-08-19T08:17:41Z)
• Robust and Controllable Object-Centric Learning through Energy-based Models [95.68748828339059]
  ours is a conceptually simple and general approach to learning object-centric representations through an energy-based model. We show that ours can be easily integrated into existing architectures and can effectively extract high-quality object-centric representations.
  arXiv  Detail & Related papers  (2022-10-11T15:11:15Z)
• Dynamic and Static Object Detection Considering Fusion Regions and Point-wise Features [7.41540085468436]
  This paper proposes a new approach to detect static and dynamic objects in front of an autonomous vehicle. Our approach can also get other characteristics from the objects detected, like their position, velocity, and heading. To demonstrate our proposal's performance, we asses it through a benchmark dataset and real-world data obtained from an autonomous platform.
  arXiv  Detail & Related papers  (2021-07-27T09:42:18Z)
• Unadversarial Examples: Designing Objects for Robust Vision [100.4627585672469]
  We develop a framework that exploits the sensitivity of modern machine learning algorithms to input perturbations in order to design "robust objects" We demonstrate the efficacy of the framework on a wide variety of vision-based tasks ranging from standard benchmarks to (in-simulation) robotics.
  arXiv  Detail & Related papers  (2020-12-22T18:26:07Z)
• Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications [77.00418462388525]
  This paper presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection.
  arXiv  Detail & Related papers  (2020-11-26T00:32:07Z)
• VATLD: A Visual Analytics System to Assess, Understand and Improve Traffic Light Detection [15.36267013724161]
  We propose a visual analytics system, VATLD, to assess, understand, and improve the accuracy and robustness of traffic light detectors in autonomous driving applications. The disentangled representation learning extracts data semantics to augment human cognition with human-friendly visual summarization. We also demonstrate the effectiveness of various performance improvement strategies with our visual analytics system, VATLD, and illustrate some practical implications for safety-critical applications in autonomous driving.
  arXiv  Detail & Related papers  (2020-09-27T22:39:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.