Evolved Neuromorphic Control for High Speed Divergence-based Landings of MAVs

• URL: http://arxiv.org/abs/2003.03118v3
• Date: Thu, 23 Jul 2020 17:13:23 GMT
• Title: Evolved Neuromorphic Control for High Speed Divergence-based Landings of MAVs
• Authors: J. J. Hagenaars, F. Paredes-Vall\'es, S. M. Boht\'e, G. C. H. E. de Croon
• Abstract summary: We develop spiking neural networks for controlling landings of micro air vehicles. We demonstrate that the resulting neuromorphic controllers transfer robustly from a simulation to the real world. To the best of our knowledge, this work is the first to integrate spiking neural networks in the control loop of a real-world flying robot.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Flying insects are capable of vision-based navigation in cluttered environments, reliably avoiding obstacles through fast and agile maneuvers, while being very efficient in the processing of visual stimuli. Meanwhile, autonomous micro air vehicles still lag far behind their biological counterparts, displaying inferior performance at a much higher energy consumption. In light of this, we want to mimic flying insects in terms of their processing capabilities, and consequently show the efficiency of this approach in the real world. This letter does so through evolving spiking neural networks for controlling landings of micro air vehicles using optical flow divergence from a downward-looking camera. We demonstrate that the resulting neuromorphic controllers transfer robustly from a highly abstracted simulation to the real world, performing fast and safe landings while keeping network spike rate minimal. Furthermore, we provide insight into the resources required for successfully solving the problem of divergence-based landing, showing that high-resolution control can be learned with only a single spiking neuron. To the best of our knowledge, this work is the first to integrate spiking neural networks in the control loop of a real-world flying robot. Videos of the experiments can be found at https://bit.ly/neuro-controller .

Related papers

• Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidance [13.467819526775472]
  We demonstrate the capabilities of an attention-based end-to-end approach for high-speed vision-based quadrotor obstacle avoidance. We train and compare convolutional, U-Net, and recurrent architectures against vision transformer (ViT) models for depth image-to-control in high-fidelity simulation.
  arXiv  Detail & Related papers  (2024-05-16T18:36:43Z)
• Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning [66.10854214036605]
  A central question in robotics is how to design a control system for an agile mobile robot. We show that a neural network controller trained with reinforcement learning (RL) outperformed optimal control (OC) methods in this setting. Our findings allowed us to push an agile drone to its maximum performance, achieving a peak acceleration greater than 12 times the gravitational acceleration and a peak velocity of 108 kilometers per hour.
  arXiv  Detail & Related papers  (2023-10-17T02:40:27Z)
• Learning and Adapting Agile Locomotion Skills by Transferring Experience [71.8926510772552]
  We propose a framework for training complex robotic skills by transferring experience from existing controllers to jumpstart learning new tasks. We show that our method enables learning complex agile jumping behaviors, navigating to goal locations while walking on hind legs, and adapting to new environments.
  arXiv  Detail & Related papers  (2023-04-19T17:37:54Z)
• Fully neuromorphic vision and control for autonomous drone flight [5.358212984063069]
  Event-based vision and spiking neural hardware promises to exhibit similar characteristics. Here, we present a fully learned neuromorphic pipeline for controlling a drone flying. Results illustrate the potential of neuromorphic sensing and processing for enabling smaller network per flight.
  arXiv  Detail & Related papers  (2023-03-15T17:19:45Z)
• Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs [69.19616451596342]
  Miniaturized unmanned aerial vehicles (UAVs) are an emerging and trending topic. We leverage a novel neural architecture search (NAS) technique to automatically identify several convolutional neural networks (CNNs) for a visual pose estimation task. Our results improve the State-of-the-Art by reducing the in-field control error of 32% while achieving a real-time onboard inference-rate of 10Hz@10mW and 50Hz@90mW.
  arXiv  Detail & Related papers  (2023-03-03T14:02:09Z)
• Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds [96.74836678572582]
  We present a learning-based approach that allows rapid online adaptation by incorporating pretrained representations through deep learning. Neural-Fly achieves precise flight control with substantially smaller tracking error than state-of-the-art nonlinear and adaptive controllers.
  arXiv  Detail & Related papers  (2022-05-13T21:55:28Z)
• Learning High-Speed Flight in the Wild [101.33104268902208]
  We propose an end-to-end approach that can autonomously fly quadrotors through complex natural and man-made environments at high speeds. The key principle is to directly map noisy sensory observations to collision-free trajectories in a receding-horizon fashion. By simulating realistic sensor noise, our approach achieves zero-shot transfer from simulation to challenging real-world environments.
  arXiv  Detail & Related papers  (2021-10-11T09:43:11Z)
• Evolved neuromorphic radar-based altitude controller for an autonomous open-source blimp [4.350434044677268]
  In this paper, we propose an evolved altitude controller based on an SNN for a robotic airship. We also present an SNN-based controller architecture, an evolutionary framework for training the network in a simulated environment, and a control strategy for ameliorating the gap with reality.
  arXiv  Detail & Related papers  (2021-10-01T20:48:43Z)
• Design and implementation of a parsimonious neuromorphic PID for onboard altitude control for MAVs using neuromorphic processors [3.7384509727711923]
  Low-level controllers are often neglected and remain outside of the neuromorphic loop. We propose a parsimonious and adjustable neuromorphic PID controller, endowed with a minimal number of 93 neurons. Our results confirm the suitability of such low-level neuromorphic controllers, ultimately with a very high update frequency.
  arXiv  Detail & Related papers  (2021-09-21T14:27:11Z)
• Event-driven Vision and Control for UAVs on a Neuromorphic Chip [41.733091458634874]
  Event-based cameras produce a sparse stream of events that can be processed more efficiently and with a lower latency than images. We show how an event-based vision algorithm can be implemented as a spiking neuronal network on a neuromorphic chip and used in a drone controller. Our spiking neuronal network on chip is the first example of a neuromorphic vision-based controller solving a high-speed UAV control task.
  arXiv  Detail & Related papers  (2021-08-08T17:46:52Z)
• Learning Quadrupedal Locomotion over Challenging Terrain [68.51539602703662]
  Legged locomotion can dramatically expand the operational domains of robotics. Conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. Here we present a radically robust controller for legged locomotion in challenging natural environments.
  arXiv  Detail & Related papers  (2020-10-21T19:11:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.