Safety-Oriented Evaluation of Language Understanding Systems for Air Traffic Control

• URL: http://arxiv.org/abs/2605.11769v1
• Date: Tue, 12 May 2026 08:39:54 GMT
• Title: Safety-Oriented Evaluation of Language Understanding Systems for Air Traffic Control
• Abstract summary: We propose a safety-oriented, consequence-aware evaluation framework tailored to Air Traffic Control (ATC) operations.<n>Our results reveal that while current LLMs achieve reasonable aggregate accuracy, their operational reliability is severely limited.<n>These findings highlight the necessity of consequence-aware evaluation protocols for the responsible deployment of AI-assisted ATC systems.
• Score: 7.102251210849638
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Air Traffic Control (ATC) is a safety-critical domain in which incorrect interpretation of instructions may lead to severe operational consequences. While large language models (LLMs) demonstrate strong general performance, their reliability in operational ATC environments remains unclear. Existing evaluation approaches, largely based on aggregate metrics such as F1 or macro accuracy, treat all errors uniformly and fail to account for the asymmetric consequences of high-risk semantic mistakes (e.g., incorrect runway identifiers or movement constraints). To address this gap, we propose a safety-oriented, consequence-aware evaluation framework tailored to ATC operations. Our results reveal that while current LLMs achieve reasonable aggregate accuracy, their operational reliability is severely limited. Evaluated on clean transcripts, the peak Risk Score reaches only 0.69, with most models scoring below 0.6 despite high macro-F1 performance. Further analysis shows that errors concentrate in high-impact entities despite relatively stable action-type classification, indicating structural grounding deficiencies. These findings highlight the necessity of consequence-aware evaluation protocols for the responsible deployment of AI-assisted ATC systems.

Related papers

• FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models [45.90905049052029]
  We propose FLY-EVAL++, an evidence-driven evaluation protocol for large language models in safety-critical environments.<n>We instantiate FLY-EVAL++ for Flight Trajectory and Attitude Prediction (FTAP) by extending the PilotBench setting with history-conditioned and multi-step prediction tasks.<n>Our results show that evaluation in safety-critical domains should measure constraint satisfaction and structured validity explicitly rather than rely on accuracy-centric reporting alone.
  arXiv  Detail & Related papers  (2026-09-03T16:00:02Z)
• Beyond Semantic Accuracy: Consequence-Aware Evaluation for Safety-Critical Language Understanding [6.5835891807589775]
  We study the problem in air traffic control, where controller-pilot communication demands near-zero error tolerance.<n>We use consequence-aware evaluation to test whether semantic scores misstate operational reliabil- ity.
  arXiv  Detail & Related papers  (2026-08-25T14:39:02Z)
• SafeAlign-VLA: A Negative-Enhanced Safe Alignment Framework for Risk-Aware Autonomous Driving [5.963051368087439]
  Vision-Language-Action (VLA) models are promising due to their strong reasoning capabilities.<n>We propose SafeAlign-VLA, a negative-enhanced safe alignment framework that incorporates negative data into supervised learning and reinforcement learning.<n>Experiments on NAVSIM and DeepAccident validate the proposed framework.
  arXiv  Detail & Related papers  (2026-05-19T08:26:43Z)
• Action-Conditioned Risk Gating for Safety-Critical Control under Partial Observability [79.08785366532287]
  We propose a lightweight risk-gated reinforcement learning approximation for risk-sensitive control under partial observability.<n>We evaluate the approach in two safety-critical partially observable domains: automated glucose regulation and safety-constrained navigation.
  arXiv  Detail & Related papers  (2026-05-14T01:23:09Z)
• SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models [46.968844120076916]
  We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards.<n>We evaluate eleven state-of-the-art models on not only hazard recognition, but also active risk mitigation through embodied planning.
  arXiv  Detail & Related papers  (2026-04-21T16:27:20Z)
• TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories [20.868825285848196]
  Safety guardrails are well-benchmarked for natural language responses, but their efficacy remains largely unexplored within multi-step tool-use trajectories.<n>To address this gap, we introduce StructuralSafe-Bench, the first comprehensive benchmark specifically designed to assess mid-trajectory safety.
  arXiv  Detail & Related papers  (2026-04-08T15:46:14Z)
• Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation [2.102846336724103]
  Procedure-Aware Evaluation (PAE) is a framework that formalizes agent procedures as structured observations.<n>We evaluate state-of-the-art Large Language Model (LLM)-based agents on tau-bench.
  arXiv  Detail & Related papers  (2026-03-03T15:47:41Z)
• Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models [62.16655896700062]
  Activation steering is a technique to enhance the utility of Large Language Models (LLMs)<n>We show that it unintentionally introduces critical and under-explored safety risks.<n>Experiments reveal that these interventions act as a force multiplier, creating new vulnerabilities to jailbreaks and increasing attack success rates to over 80% on standard benchmarks.
  arXiv  Detail & Related papers  (2026-02-03T12:32:35Z)
• Accurate Failure Prediction in Agents Does Not Imply Effective Failure Prevention [2.0838114053126366]
  Proactive interventions by LLM critic models are often assumed to improve reliability, yet their effects at deployment time are poorly understood.<n>We show that a binary LLM critic with strong offline accuracy (AUROC 0.94) can nevertheless cause severe performance degradation.<n>We propose a pre-deployment test that uses a small pilot of 50 tasks to estimate whether intervention is likely to help or harm, without requiring full deployment.
  arXiv  Detail & Related papers  (2026-02-03T10:02:50Z)
• Mitigating Safety Tax via Distribution-Grounded Refinement in Large Reasoning Models [63.368505631152594]
  Safety alignment incurs safety tax that perturbs a large reasoning model's (LRM) general reasoning ability.<n>Existing datasets used for safety alignment for an LRM are usually constructed by distilling safety reasoning traces and answers from an external LRM or human labeler.<n>We propose a safety alignment dataset construction method, dubbed DGR. DGR transforms and refines an existing out-of-distributional safety reasoning dataset to be aligned with the target's LLM inner distribution.
  arXiv  Detail & Related papers  (2026-02-02T14:18:48Z)
• Trajectory Guard -- A Lightweight, Sequence-Aware Model for Real-Time Anomaly Detection in Agentic AI [0.0]
  Trajectory Guard is a Siamese Recurrent Autoencoder with a hybrid loss function that jointly learns task-trajectory alignment via contrastive learning and sequential validity via reconstruction.<n>At 32 ms latency, our approach runs 17$-27times$ faster than LLM Judge baselines, enabling real-time safety verification in production deployments.
  arXiv  Detail & Related papers  (2026-01-02T00:27:11Z)
• SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior [56.10557932893919]
  We present SafetyAnalyst, a novel AI safety moderation framework.<n>Given an AI behavior, SafetyAnalyst uses chain-of-thought reasoning to analyze its potential consequences.<n>It aggregates effects into a harmfulness score using 28 fully interpretable weight parameters.
  arXiv  Detail & Related papers  (2024-10-22T03:38:37Z)
• Safety Margins for Reinforcement Learning [53.10194953873209]
  We show how to leverage proxy criticality metrics to generate safety margins. We evaluate our approach on learned policies from APE-X and A3C within an Atari environment.
  arXiv  Detail & Related papers  (2023-07-25T16:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.