Neuromorphic Attitude Estimation and Control

• URL: http://arxiv.org/abs/2411.13945v1
• Date: Thu, 21 Nov 2024 08:54:45 GMT
• Title: Neuromorphic Attitude Estimation and Control
• Authors: Stein Stroobants, Christophe de Wagter, Guido C. H. E. De Croon,
• Abstract summary: This research presents the first neuromorphic control system using a spiking neural network (SNN) We apply this method to low-level attitude estimation and control for a quadrotor, deploying the SNN on a tiny Crazyflie. Our work shows the feasibility of performing neuromorphic end-to-end control, laying the basis for highly energy-efficient and low-latency neuromorphic autopilots.
• Score: 17.895261339368815
• License:
• Abstract: The real-world application of small drones is mostly hampered by energy limitations. Neuromorphic computing promises extremely energy-efficient AI for autonomous flight, but is still challenging to train and deploy on real robots. In order to reap the maximal benefits from neuromorphic computing, it is desired to perform all autonomy functions end-to-end on a single neuromorphic chip, from low-level attitude control to high-level navigation. This research presents the first neuromorphic control system using a spiking neural network (SNN) to effectively map a drone's raw sensory input directly to motor commands. We apply this method to low-level attitude estimation and control for a quadrotor, deploying the SNN on a tiny Crazyflie. We propose a modular SNN, separately training and then merging estimation and control sub-networks. The SNN is trained with imitation learning, using a flight dataset of sensory-motor pairs. Post-training, the network is deployed on the Crazyflie, issuing control commands from sensor inputs at $500$Hz. Furthermore, for the training procedure we augmented training data by flying a controller with additional excitation and time-shifting the target data to enhance the predictive capabilities of the SNN. On the real drone the perception-to-control SNN tracks attitude commands with an average error of $3$ degrees, compared to $2.5$ degrees for the regular flight stack. We also show the benefits of the proposed learning modifications for reducing the average tracking error and reducing oscillations. Our work shows the feasibility of performing neuromorphic end-to-end control, laying the basis for highly energy-efficient and low-latency neuromorphic autopilots.

Related papers

• On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors [53.59319391812798]
  Sub-SI50gram nano-drones are gaining momentum in both academia and industry. Their most compelling applications rely on onboard deep learning models for perception. When deployed in unknown environments, these models often underperform due to domain shift. We propose for the first time, on-device learning aboard nano-drones, where the first part of the in-field mission is dedicated to self-supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-03-06T22:04:14Z)
• Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning [51.386945803485084]
  We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer. To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
  arXiv  Detail & Related papers  (2024-01-09T07:31:34Z)
• Fully neuromorphic vision and control for autonomous drone flight [5.358212984063069]
  Event-based vision and spiking neural hardware promises to exhibit similar characteristics. Here, we present a fully learned neuromorphic pipeline for controlling a drone flying. Results illustrate the potential of neuromorphic sensing and processing for enabling smaller network per flight.
  arXiv  Detail & Related papers  (2023-03-15T17:19:45Z)
• Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs [69.19616451596342]
  Miniaturized unmanned aerial vehicles (UAVs) are an emerging and trending topic. We leverage a novel neural architecture search (NAS) technique to automatically identify several convolutional neural networks (CNNs) for a visual pose estimation task. Our results improve the State-of-the-Art by reducing the in-field control error of 32% while achieving a real-time onboard inference-rate of 10Hz@10mW and 50Hz@90mW.
  arXiv  Detail & Related papers  (2023-03-03T14:02:09Z)
• Training Spiking Neural Networks with Local Tandem Learning [96.32026780517097]
  Spiking neural networks (SNNs) are shown to be more biologically plausible and energy efficient than their predecessors. In this paper, we put forward a generalized learning rule, termed Local Tandem Learning (LTL) We demonstrate rapid network convergence within five training epochs on the CIFAR-10 dataset while having low computational complexity.
  arXiv  Detail & Related papers  (2022-10-10T10:05:00Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)
• Neural Moving Horizon Estimation for Robust Flight Control [6.023276947115864]
  Estimating and reacting to external disturbances is crucial for robust flight control of quadrotors. We propose a neural moving horizon estimator (NeuroMHE) that can automatically tune the MHE parameters modeled by a neural network. NeuroMHE outperforms the state-of-the-art estimator with force estimation error reductions of up to 49.4%.
  arXiv  Detail & Related papers  (2022-06-21T13:43:24Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Evolved neuromorphic radar-based altitude controller for an autonomous open-source blimp [4.350434044677268]
  In this paper, we propose an evolved altitude controller based on an SNN for a robotic airship. We also present an SNN-based controller architecture, an evolutionary framework for training the network in a simulated environment, and a control strategy for ameliorating the gap with reality.
  arXiv  Detail & Related papers  (2021-10-01T20:48:43Z)
• Design and implementation of a parsimonious neuromorphic PID for onboard altitude control for MAVs using neuromorphic processors [3.7384509727711923]
  Low-level controllers are often neglected and remain outside of the neuromorphic loop. We propose a parsimonious and adjustable neuromorphic PID controller, endowed with a minimal number of 93 neurons. Our results confirm the suitability of such low-level neuromorphic controllers, ultimately with a very high update frequency.
  arXiv  Detail & Related papers  (2021-09-21T14:27:11Z)
• Online-Learning Deep Neuro-Adaptive Dynamic Inversion Controller for Model Free Control [1.3764085113103217]
  A neuro-adaptive controller is implemented featuring a deep neural network trained on a new weight update law. The controller is able to learn the nonlinear plant quickly and displays good performance in the tracking control problem.
  arXiv  Detail & Related papers  (2021-07-21T22:46:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.