AIRHILT: A Human-in-the-Loop Testbed for Multimodal Conflict Detection in Aviation

• URL: http://arxiv.org/abs/2511.18718v1
• Date: Mon, 24 Nov 2025 03:18:55 GMT
• Title: AIRHILT: A Human-in-the-Loop Testbed for Multimodal Conflict Detection in Aviation
• Abstract summary: We introduce AIRHILT (Aviation Integrated Reasoning, Human-in-the-Loop Testbed)<n>A modular and lightweight simulation environment designed to evaluate multimodal pilot and air traffic control assistance systems for aviation conflict detection.<n>Built on the open-source Godot engine, AIRHILT synchronizes pilot and ATC radio communications, visual scene understanding from camera streams, and ADS-B surveillance data within a unified platform.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce AIRHILT (Aviation Integrated Reasoning, Human-in-the-Loop Testbed), a modular and lightweight simulation environment designed to evaluate multimodal pilot and air traffic control (ATC) assistance systems for aviation conflict detection. Built on the open-source Godot engine, AIRHILT synchronizes pilot and ATC radio communications, visual scene understanding from camera streams, and ADS-B surveillance data within a unified, scalable platform. The environment supports pilot- and controller-in-the-loop interactions, providing a comprehensive scenario suite covering both terminal area and en route operational conflicts, including communication errors and procedural mistakes. AIRHILT offers standardized JSON-based interfaces that enable researchers to easily integrate, swap, and evaluate automatic speech recognition (ASR), visual detection, decision-making, and text-to-speech (TTS) models. We demonstrate AIRHILT through a reference pipeline incorporating fine-tuned Whisper ASR, YOLO-based visual detection, ADS-B-based conflict logic, and GPT-OSS-20B structured reasoning, and present preliminary results from representative runway-overlap scenarios, where the assistant achieves an average time-to-first-warning of approximately 7.7 s, with average ASR and vision latencies of approximately 5.9 s and 0.4 s, respectively. The AIRHILT environment and scenario suite are openly available, supporting reproducible research on multimodal situational awareness and conflict detection in aviation; code and scenarios are available at https://github.com/ogarib3/airhilt.

Related papers

• SCOPE: A Lightweight-training LLM Framework for Air Traffic Control Readback Monitoring [8.082279149607116]
  Air Traffic Control (ATC) voice instructions are a primary safeguard against miscommunication in air transportation.<n>Readback anomalies remain implicated in approximately 80% of aviation incidents.<n>Traditional rule-based and machine learning approaches struggle to generalize across the highly variable and evolving phraseology of ATC communications.<n>We propose a novel lightweight-training LLM framework that advances both the efficiency and accuracy of machine-based ATC readback monitoring.
  arXiv  Detail & Related papers  (2026-05-28T07:56:24Z)
• Towards Automated Air Traffic Safety Assessment Around Non-Towered Airports Using Large Language Models [37.909132464953196]
  We investigate frameworks for post-flight safety analysis at non-towered airports using large language models (LLMs)<n>We propose a general vision-language model (VLM) approach to analyze the transcribed CTAF radio communications in natural language.<n>We qualitatively evaluate our framework on real flight data using Gemini 2.5 Pro, demonstrating accurate identification of a right-of-way violation.
  arXiv  Detail & Related papers  (2026-05-12T16:15:15Z)
• AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarios [64.51320327698231]
  We introduce AerialMind, the first large-scale RMOT benchmark in UAV scenarios.<n>We develop an innovative semi-automated collaborative agent-based labeling assistant framework.<n>We also propose HawkEyeTrack, a novel method that collaboratively enhances vision-language representation learning.
  arXiv  Detail & Related papers  (2025-11-26T04:44:27Z)
• ASTREA: Introducing Agentic Intelligence for Orbital Thermal Autonomy [51.56484100374058]
  ASTREA is the first agentic system executed on flight-heritage hardware for autonomous spacecraft operations.<n>We integrate a resource-constrained Large Language Model (LLM) agent with a reinforcement learning controller in an asynchronous architecture tailored for space-qualified platforms.
  arXiv  Detail & Related papers  (2025-09-16T08:52:13Z)
• Combining Textual and Spectral Features for Robust Classification of Pilot Communications [2.9346090159291283]
  This paper presents a novel dual pipeline machine learning framework that classifies pilot radio communications using both textual and spectral features.<n>To our knowledge, this is the first system to classify operational aircraft intent using a dual-pipeline ML framework on real-world air traffic audio.
  arXiv  Detail & Related papers  (2025-09-11T16:43:10Z)
• Air Traffic Controller Task Demand via Graph Neural Networks: An Interpretable Approach to Airspace Complexity [0.00448108533967507]
  Real-time assessment of near-term Air Traffic Controller (ATCO) task demand is a critical challenge in an increasingly crowded airspace.<n>This work introduces an interpretable Graph Neural Network (GNN) framework to address this gap.<n>We derive an interpretable, per-aircraft task demand score by systematically ablating aircraft and measuring the impact on the model's predictions.
  arXiv  Detail & Related papers  (2025-07-17T17:02:42Z)
• AeroLite-MDNet: Lightweight Multi-task Deviation Detection Network for UAV Landing [9.858832286469765]
  We propose a deviation warning system for UAV landings powered by a novel vision-based model called AeroLite-MDNet.<n>We introduce a new evaluation metric, Average Warning Delay (AWD), to quantify the system's sensitivity to landing deviations.<n> Experimental results show that our system achieves an AWD of 0.7 seconds with a deviation detection accuracy of 98.6%.
  arXiv  Detail & Related papers  (2025-06-25T13:48:30Z)
• Griffin: Aerial-Ground Cooperative Detection and Tracking Dataset and Benchmark [15.405137983083875]
  Aerial-ground cooperation offers a promising solution by integrating UAVs' aerial views with ground vehicles' local observations.<n>This paper presents a comprehensive solution for aerial-ground cooperative 3D perception through three key contributions.
  arXiv  Detail & Related papers  (2025-03-10T07:00:07Z)
• Angle Robustness Unmanned Aerial Vehicle Navigation in GNSS-Denied Scenarios [66.05091704671503]
  We present a novel angle navigation paradigm to deal with flight deviation in point-to-point navigation tasks. We also propose a model that includes the Adaptive Feature Enhance Module, Cross-knowledge Attention-guided Module and Robust Task-oriented Head Module.
  arXiv  Detail & Related papers  (2024-02-04T08:41:20Z)
• Efficient Real-time Smoke Filtration with 3D LiDAR for Search and Rescue with Autonomous Heterogeneous Robotic Systems [56.838297900091426]
  Smoke and dust affect the performance of any mobile robotic platform due to their reliance on onboard perception systems. This paper proposes a novel modular computation filtration pipeline based on intensity and spatial information.
  arXiv  Detail & Related papers  (2023-08-14T16:48:57Z)
• Convergence of Communications, Control, and Machine Learning for Secure and Autonomous Vehicle Navigation [78.60496411542549]
  Connected and autonomous vehicles (CAVs) can reduce human errors in traffic accidents, increase road efficiency, and execute various tasks. Reaping these benefits requires CAVs to autonomously navigate to target destinations. This article proposes solutions using the convergence of communication theory, control theory, and machine learning to enable effective and secure CAV navigation.
  arXiv  Detail & Related papers  (2023-07-05T21:38:36Z)
• AVOIDDS: Aircraft Vision-based Intruder Detection Dataset and Simulator [37.579437595742995]
  We introduce AVOIDDS, a realistic object detection benchmark for the vision-based aircraft detect-and-avoid problem. We provide a labeled dataset consisting of 72,000 photorealistic images of intruder aircraft with various lighting conditions. We also provide an interface that evaluates trained models on slices of this dataset to identify changes in performance with respect to changing environmental conditions.
  arXiv  Detail & Related papers  (2023-06-19T23:58:07Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• A Multi-UAV System for Exploration and Target Finding in Cluttered and GPS-Denied Environments [68.31522961125589]
  We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Results indicate that the proposed multi-UAV system has improvements in terms of time-cost, the proportion of search area surveyed, as well as successful rates for search and rescue missions.
  arXiv  Detail & Related papers  (2021-07-19T12:54:04Z)
• A Simplified Framework for Air Route Clustering Based on ADS-B Data [0.0]
  This paper presents a framework that can support to detect the typical air routes between airports based on ADS-B data. As a matter of fact, our framework can be taken into account to reduce practically the computational cost for air flow optimization.
  arXiv  Detail & Related papers  (2021-07-07T08:55:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.