All Eyes, no IMU: Learning Flight Attitude from Vision Alone

• URL: http://arxiv.org/abs/2507.11302v1
• Date: Tue, 15 Jul 2025 13:31:27 GMT
• Title: All Eyes, no IMU: Learning Flight Attitude from Vision Alone
• Authors: Jesse J. Hagenaars, Stein Stroobants, Sander M. Bohte, Guido C. H. E. De Croon,
• Abstract summary: We present the first vision-only approach to flight control for use in generic environments.<n>We show that a quadrotor drone equipped with a downward-facing event camera can estimate its attitude and rotation rate from just the event stream.<n>Our work showcases vision-only flight control as a promising candidate for enabling autonomous, insect-scale flying robots.
• Score: 13.780259242770724
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision is an essential part of attitude control for many flying animals, some of which have no dedicated sense of gravity. Flying robots, on the other hand, typically depend heavily on accelerometers and gyroscopes for attitude stabilization. In this work, we present the first vision-only approach to flight control for use in generic environments. We show that a quadrotor drone equipped with a downward-facing event camera can estimate its attitude and rotation rate from just the event stream, enabling flight control without inertial sensors. Our approach uses a small recurrent convolutional neural network trained through supervised learning. Real-world flight tests demonstrate that our combination of event camera and low-latency neural network is capable of replacing the inertial measurement unit in a traditional flight control loop. Furthermore, we investigate the network's generalization across different environments, and the impact of memory and different fields of view. While networks with memory and access to horizon-like visual cues achieve best performance, variants with a narrower field of view achieve better relative generalization. Our work showcases vision-only flight control as a promising candidate for enabling autonomous, insect-scale flying robots.

Related papers

• Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidance [13.467819526775472]
  We demonstrate the capabilities of an attention-based end-to-end approach for high-speed vision-based quadrotor obstacle avoidance.<n>We train and compare convolutional, U-Net, and recurrent architectures against vision transformer (ViT) models for depth image-to-control in high-fidelity simulation.
  arXiv  Detail & Related papers  (2024-05-16T18:36:43Z)
• Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing [52.50284630866713]
  Existing systems often require hand-engineered components for state estimation, planning, and control. This paper tackles the vision-based autonomous-drone-racing problem by learning deep sensorimotor policies.
  arXiv  Detail & Related papers  (2022-10-26T19:03:17Z)
• Learning a Single Near-hover Position Controller for Vastly Different Quadcopters [56.37274861303324]
  This paper proposes an adaptive near-hover position controller for quadcopters. It can be deployed to quadcopters of very different mass, size and motor constants. It also shows rapid adaptation to unknown disturbances during runtime.
  arXiv  Detail & Related papers  (2022-09-19T17:55:05Z)
• Masked Visual Pre-training for Motor Control [118.18189211080225]
  Self-supervised visual pre-training from real-world images is effective for learning motor control tasks from pixels. We freeze the visual encoder and train neural network controllers on top with reinforcement learning. This is the first self-supervised model to exploit real-world images at scale for motor control.
  arXiv  Detail & Related papers  (2022-03-11T18:58:10Z)
• Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents [45.36060508554703]
  Humans race drones faster than neural networks trained for end-to-end autonomous flight. This work investigates whether neural networks capable of imitating human eye gaze behavior and attention can improve neural network performance.
  arXiv  Detail & Related papers  (2022-01-07T18:07:51Z)
• Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations [75.60524561611008]
  This work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate.
  arXiv  Detail & Related papers  (2021-09-28T20:25:10Z)
• Visual Navigation Among Humans with Optimal Control as a Supervisor [72.5188978268463]
  We propose an approach that combines learning-based perception with model-based optimal control to navigate among humans. Our approach is enabled by our novel data-generation tool, HumANav. We demonstrate that the learned navigation policies can anticipate and react to humans without explicitly predicting future human motion.
  arXiv  Detail & Related papers  (2020-03-20T16:13:47Z)
• Evolved Neuromorphic Control for High Speed Divergence-based Landings of MAVs [0.0]
  We develop spiking neural networks for controlling landings of micro air vehicles. We demonstrate that the resulting neuromorphic controllers transfer robustly from a simulation to the real world. To the best of our knowledge, this work is the first to integrate spiking neural networks in the control loop of a real-world flying robot.
  arXiv  Detail & Related papers  (2020-03-06T10:19:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.