Autonomous Driving using Spiking Neural Networks on Dynamic Vision Sensor Data: A Case Study of Traffic Light Change Detection

• URL: http://arxiv.org/abs/2311.09225v1
• Date: Wed, 27 Sep 2023 23:31:30 GMT
• Title: Autonomous Driving using Spiking Neural Networks on Dynamic Vision Sensor Data: A Case Study of Traffic Light Change Detection
• Authors: Xuelei Chen
• Abstract summary: Spiking neural networks (SNNs) provide an alternative model to process information and make decisions. Recent work using SNNs for autonomous driving mostly focused on simple tasks like lane keeping in simplified simulation environments. This project studies SNNs on photo-realistic driving scenes in the CARLA simulator, which is an important step toward using SNNs on real vehicles.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous driving is a challenging task that has gained broad attention from both academia and industry. Current solutions using convolutional neural networks require large amounts of computational resources, leading to high power consumption. Spiking neural networks (SNNs) provide an alternative computation model to process information and make decisions. This biologically plausible model has the advantage of low latency and energy efficiency. Recent work using SNNs for autonomous driving mostly focused on simple tasks like lane keeping in simplified simulation environments. This project studies SNNs on photo-realistic driving scenes in the CARLA simulator, which is an important step toward using SNNs on real vehicles. The efficacy and generalizability of the method will be investigated.

Related papers

• DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach [49.56404236394601]
  We formulate the problem of joint DNN partitioning, task offloading, and resource allocation in Vehicular Edge Computing. Our objective is to minimize the DNN-based task completion time while guaranteeing the system stability over time. We propose a Multi-Agent Diffusion-based Deep Reinforcement Learning (MAD2RL) algorithm, incorporating the innovative use of diffusion models.
  arXiv  Detail & Related papers  (2024-06-11T06:31:03Z)
• PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search [64.28335667655129]
  Multiple object tracking is a critical task in autonomous driving. As tracking accuracy improves, neural networks become increasingly complex, posing challenges for their practical application in real driving scenarios due to the high level of latency. In this paper, we explore the use of the neural architecture search (NAS) methods to search for efficient architectures for tracking, aiming for low real-time latency while maintaining relatively high accuracy.
  arXiv  Detail & Related papers  (2024-03-23T04:18:49Z)
• Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning [51.386945803485084]
  We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer. To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
  arXiv  Detail & Related papers  (2024-01-09T07:31:34Z)
• LaneSNNs: Spiking Neural Networks for Lane Detection on the Loihi Neuromorphic Processor [12.47874622269824]
  We present a new SNN-based approach, called LaneSNN, for detecting the lanes marked on the streets using the event-based camera input. We implement and map the learned SNNs models onto the Intel Loihi Neuromorphic Research Chip. For the loss function, we develop a novel method based on the linear composition of Weighted binary Cross Entropy (WCE) and Mean Squared Error (MSE) measures.
  arXiv  Detail & Related papers  (2022-08-03T14:51:15Z)
• SpikiLi: A Spiking Simulation of LiDAR based Real-time Object Detection for Autonomous Driving [0.0]
  Spiking Neural Networks are a new neural network design approach that promises tremendous improvements in power efficiency, computation efficiency, and processing latency. We first illustrate the applicability of spiking neural networks to a complex deep learning task namely Lidar based 3D object detection for automated driving.
  arXiv  Detail & Related papers  (2022-06-06T20:05:17Z)
• Efficient Federated Learning with Spike Neural Networks for Traffic Sign Recognition [70.306089187104]
  We introduce powerful Spike Neural Networks (SNNs) into traffic sign recognition for energy-efficient and fast model training. Numerical results indicate that the proposed federated SNN outperforms traditional federated convolutional neural networks in terms of accuracy, noise immunity, and energy efficiency as well.
  arXiv  Detail & Related papers  (2022-05-28T03:11:48Z)
• Object Detection with Spiking Neural Networks on Automotive Event Data [0.0]
  We propose to train spiking neural networks (SNNs) directly on data coming from event cameras to design fast and efficient automotive embedded applications. In this paper, we conducted experiments on two automotive event datasets, establishing new state-of-the-art classification results for spiking neural networks.
  arXiv  Detail & Related papers  (2022-05-09T14:39:47Z)
• CarSNN: An Efficient Spiking Neural Network for Event-Based Autonomous Cars on the Loihi Neuromorphic Research Processor [15.093607722961407]
  Spiking Neural Networks (SNNs) can achieve high performance with low latency and low power consumption. In this paper, we use an SNN connected to an event-based camera for facing one of the key problems for Autonomous Driving (AD) Experiments are made following an offline supervised learning rule, followed by mapping the learnt SNN model on the Intel Loihi Neuromorphic Research Chip.
  arXiv  Detail & Related papers  (2021-07-01T12:20:48Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Temporal Pulses Driven Spiking Neural Network for Fast Object Recognition in Autonomous Driving [65.36115045035903]
  We propose an approach to address the object recognition problem directly with raw temporal pulses utilizing the spiking neural network (SNN) Being evaluated on various datasets, our proposed method has shown comparable performance as the state-of-the-art methods, while achieving remarkable time efficiency.
  arXiv  Detail & Related papers  (2020-01-24T22:58:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.