HoverAI: An Embodied Aerial Agent for Natural Human-Drone Interaction

• URL: http://arxiv.org/abs/2601.13801v1
• Date: Tue, 20 Jan 2026 09:59:49 GMT
• Title: HoverAI: An Embodied Aerial Agent for Natural Human-Drone Interaction
• Abstract summary: We present HoverAI, an embodied aerial agent that integrates drone mobility, infrastructure-independent visual projection, and real-time conversational AI into a unified platform.<n>HoverAI perceives users through vision and voice, responding via lip-synced avatars that adapt appearance to user demographics.
• Score: 2.6822193190929444
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Drones operating in human-occupied spaces suffer from insufficient communication mechanisms that create uncertainty about their intentions. We present HoverAI, an embodied aerial agent that integrates drone mobility, infrastructure-independent visual projection, and real-time conversational AI into a unified platform. Equipped with a MEMS laser projector, onboard semi-rigid screen, and RGB camera, HoverAI perceives users through vision and voice, responding via lip-synced avatars that adapt appearance to user demographics. The system employs a multimodal pipeline combining VAD, ASR (Whisper), LLM-based intent classification, RAG for dialogue, face analysis for personalization, and voice synthesis (XTTS v2). Evaluation demonstrates high accuracy in command recognition (F1: 0.90), demographic estimation (gender F1: 0.89, age MAE: 5.14 years), and speech transcription (WER: 0.181). By uniting aerial robotics with adaptive conversational AI and self-contained visual output, HoverAI introduces a new class of spatially-aware, socially responsive embodied agents for applications in guidance, assistance, and human-centered interaction.

Related papers

• Harness Robotic OS: A Unified Embodied-Agent Runtime for Closed-Loop Quadruped Inspection [4.737450468943755]
  This paper presents textitHarness Robotic OS (HROS), a unified embodied-agent runtime, and Argos, its realization for residential-community inspection.<n>HROS organizes the system into robot runtime, embodied autonomy skills, cognitive agent, and interaction and operations planes.<n>Experiments in a residential property environment achieved 100% waypoint reachability, outdoor localization error below 10cm, local obstacle-response latency below 200ms, and 99% success in alarm delivery.
  arXiv  Detail & Related papers  (2026-09-10T08:25:46Z)
• OmniAI: A Surface-Adaptive Aerial Projection Interface for Human--Drone Interaction [2.6822193190929444]
  We present OmniAI, an aerial agent that supports surface-adaptive interaction by switching projection between an onboard screen and nearby surfaces.<n>Projection surfaces are detected online using RGB-D sensing and RANSAC plane fitting, without pre-mapped geometry.
  arXiv  Detail & Related papers  (2026-08-01T15:41:36Z)
• ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI [110.18670135256235]
  Embodied AI is moving from isolated perception or action modules toward physical agents that understand, plan under goals, monitor progress, and improve from experience.<n>We present ACE-Brain-0.5, a unified embodied foundation model that organizes robot intelligence into five coupled functions.<n>A single 8B backbone instantiates the first four functions: grounding objects and affordances, reasoning over 3D and egocentric spatial relations, decomposing instructions into subgoals, generating navigation and manipulation actions, and estimating progress for verification and recovery.
  arXiv  Detail & Related papers  (2026-07-05T17:43:06Z)
• AURA: Multimodal Shared Autonomy for Real-World Urban Navigation [49.419374037785815]
  Long-horizon navigation in complex urban environments relies heavily on continuous human operation.<n>We present Assistive Urban Robot Autonomy (AURA), a new multi-modal framework that decomposes urban navigation into high-level human instruction and low-level AI control.<n>AURA effectively follows human instructions, reduces manual operation effort, and improves navigation stability, while enabling online adaptation.
  arXiv  Detail & Related papers  (2026-04-02T05:59:31Z)
• Robotic Grasping and Placement Controlled by EEG-Based Hybrid Visual and Motor Imagery [64.82869118243723]
  We present a framework that integrates EEG-based visual and motor imagery (VI/MI) with robotic control to enable real-time, intention-driven grasping and placement.<n>Motivated by the promise of BCI-driven robotics, this system bridges neural signals with physical control by deploying offline-pretrained decoders in a zero-shot manner.
  arXiv  Detail & Related papers  (2026-03-03T17:41:42Z)
• An Approach to Combining Video and Speech with Large Language Models in Human-Robot Interaction [0.0]
  This work presents a novel HRI framework that combines advanced vision-language models, speech processing, and fuzzy logic.<n>The proposed system integrates Florence-2 for object detection, Llama 3.1 for natural language understanding, and Whisper for speech recognition.<n> Experimental evaluations conducted on consumer-grade hardware demonstrate a command execution accuracy of 75%.
  arXiv  Detail & Related papers  (2026-02-23T09:05:15Z)
• DroneVLA: VLA based Aerial Manipulation [2.1645011609137295]
  This work introduces a novel concept of autonomous aerial manipulation system capable of interpreting high-level natural language commands to retrieve objects and deliver them to a human user.<n>The system is intended to integrate a MediaPipe based on Grounding DINO and a Vision-Language-Action model with a custom-built drone equipped with a 1-DOF gripper and an Intel RealSense RGB-D camera.<n>We demonstrate the system's efficacy through real-world experiments for localization and navigation, which resulted in a 0.164m, 0.070m, and 0.084m of max, mean euclidean, and root-mean squared
  arXiv  Detail & Related papers  (2026-01-20T10:08:00Z)
• The ICASSP 2026 HumDial Challenge: Benchmarking Human-like Spoken Dialogue Systems in the LLM Era [95.35748535806744]
  We launch the first Human-like Spoken Dialogue Systems Challenge (HumDial) at ICASSP 2026.<n>This paper summarizes the dataset, track configurations, and the final results.
  arXiv  Detail & Related papers  (2026-01-09T06:32:30Z)
• Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions [110.43343503158306]
  This paper embeds the manual-assisted task into a vision-language-action framework, where the assistant provides services to the instructor following egocentric vision and commands.<n>Under this setting, we accomplish InterVLA, the first large-scale human-object-human interaction dataset with 11.4 hours and 1.2M frames of multimodal data.<n>We establish novel benchmarks on egocentric human motion estimation, interaction synthesis, and interaction prediction with comprehensive analysis.
  arXiv  Detail & Related papers  (2025-08-06T17:46:23Z)
• Recognizing Actions from Robotic View for Natural Human-Robot Interaction [52.00935005918032]
  Natural Human-Robot Interaction (N-HRI) requires robots to recognize human actions at varying distances and states, regardless of whether the robot itself is in motion or stationary.<n>Existing benchmarks for N-HRI fail to address the unique complexities in N-HRI due to limited data, modalities, task categories, and diversity of subjects and environments.<n>We introduce (Action from Robotic View) a large-scale dataset for perception-centric robotic views prevalent in mobile service robots.
  arXiv  Detail & Related papers  (2025-07-30T09:48:34Z)
• Multi-model fusion for Aerial Vision and Dialog Navigation based on human attention aids [69.98258892165767]
  We present an aerial navigation task for the 2023 ICCV Conversation History. We propose an effective method of fusion training of Human Attention Aided Transformer model (HAA-Transformer) and Human Attention Aided LSTM (HAA-LSTM) models.
  arXiv  Detail & Related papers  (2023-08-27T10:32:52Z)
• Can Visual Context Improve Automatic Speech Recognition for an Embodied Agent? [3.7311680121118345]
  We propose a new decoder biasing technique to incorporate the visual context while ensuring the ASR output does not degrade for incorrect context. We achieve a 59% relative reduction in WER from an unmodified ASR system.
  arXiv  Detail & Related papers  (2022-10-21T11:16:05Z)
• Self-supervised reinforcement learning for speaker localisation with the iCub humanoid robot [58.2026611111328]
  Looking at a person's face is one of the mechanisms that humans rely on when it comes to filtering speech in noisy environments. Having a robot that can look toward a speaker could benefit ASR performance in challenging environments. We propose a self-supervised reinforcement learning-based framework inspired by the early development of humans.
  arXiv  Detail & Related papers  (2020-11-12T18:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.