Evaluating Explainability in Safety-Critical ATR Systems: Limitations of Post-Hoc Methods and Paths Toward Robust XAI

• URL: http://arxiv.org/abs/2605.05748v1
• Date: Thu, 07 May 2026 06:42:08 GMT
• Title: Evaluating Explainability in Safety-Critical ATR Systems: Limitations of Post-Hoc Methods and Paths Toward Robust XAI
• Abstract summary: This paper presents a structured evaluation of explainability methods in the context of safety-critical ATR systems.<n>We identify major XAI paradigms, including saliency-based, attention-based, and surrogate ap proaches.<n>We derive critical failure modes such as spurious explanations, instability under perturbations, and overtrust induced by visually con vincing outputs.
• Score: 2.3099144596725574
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Explainable Artificial Intelligence (XAI) is increasingly rec ognized as essential for deploying machine learning systems in safety critical environments. In Automatic Target Recognition (ATR), where models operate on image, video, radar, and multisensor data, high pre dictive performance alone is insufficient. Model decisions must also be interpretable, reliable, and suitable for validation. This paper presents a structured evaluation of explainability methods in the context of safety-critical ATR systems: We identify major XAI paradigms, including saliency-based, attention-based, and surrogate ap proaches, as well as recent detection-aware extensions. Based on this, we formalize explainability as an assurance-oriented assessment problem, introduce a taxonomy, and assess these methods with respect to four key dimensions: interpretability, robustness, vulnerability to manipula tion, and suitability for validation and verification. The analysis identifies systematic limitations of current post-hoc explanation methods. In par ticular, we derive critical failure modes such as spurious explanations, instability under perturbations, and overtrust induced by visually con vincing outputs. These findings indicate that widely used XAI techniques may be insufficient for safety-critical deployment. Finally, we discuss implications for ATR systems and outline directions toward more robust, causally grounded, and physically informed explain ability methods. Our results emphasize the need to move beyond visually plausible explanations toward approaches that support reliable decision making and system-level assurance.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• A Bayesian Framework for Uncertainty-Aware Explanations in Power Quality Disturbance Classification [3.68407754316989]
  This paper proposes a Bayesian explanation framework that models explanation uncertainty by generating a relevance attribution distribution for each instance.<n>It allows experts to select explanations based on confidence percentiles, thereby tailoring interpretability according to specific disturbance types.<n>Experiments on synthetic and real-world power quality datasets demonstrate that the proposed framework improves the transparency and reliability of PQD classifiers.
  arXiv  Detail & Related papers  (2026-04-15T09:22:47Z)
• From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
  This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
  arXiv  Detail & Related papers  (2026-01-22T06:21:31Z)
• AI-Generated Image Detection: An Empirical Study and Future Research Directions [6.891145787446519]
  Threats posed by AI-generated media, particularly deepfakes, are raising significant challenges for forensics.<n>Several forensic methods have been proposed, they suffer from three critical gaps.<n>These limitations hinder fair comparison, obscure true robustness, and restrict deployment in security-critical applications.
  arXiv  Detail & Related papers  (2025-11-04T18:13:48Z)
• Towards Robust Artificial Intelligence: Self-Supervised Learning Approach for Out-of-Distribution Detection [0.19599274203282294]
  This paper proposes an approach to improve OOD detection without the need of labeled data.<n>The proposed approach leverages the principles of self-supervised learning, allowing the model to learn useful representations from unlabeled data.
  arXiv  Detail & Related papers  (2025-10-14T16:55:25Z)
• Deep Learning Models for Robust Facial Liveness Detection [56.08694048252482]
  This study introduces a robust solution through novel deep learning models addressing the deficiencies in contemporary anti-spoofing techniques.<n>By innovatively integrating texture analysis and reflective properties associated with genuine human traits, our models distinguish authentic presence from replicas with remarkable precision.
  arXiv  Detail & Related papers  (2025-08-12T17:19:20Z)
• Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods [0.0]
  This literature review consolidates the rapidly evolving field of AI safety evaluations.<n>It proposes a systematic taxonomy around three dimensions: what properties we measure, how we measure them, and how these measurements integrate into frameworks.
  arXiv  Detail & Related papers  (2025-05-08T16:55:07Z)
• Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling [41.19330514054401]
  Large language models (LLMs) are prone to hallucination stemming from misaligned self-awareness.<n>We propose the Explicit Knowledge Boundary Modeling framework to integrate fast and slow reasoning systems to harmonize reliability and usability.
  arXiv  Detail & Related papers  (2025-03-04T03:16:02Z)
• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• Investigating Robustness in Cyber-Physical Systems: Specification-Centric Analysis in the face of System Deviations [8.8690305802668]
  A critical attribute of cyber-physical systems (CPS) is robustness, denoting its capacity to operate safely. This paper proposes a novel specification-based robustness, which characterizes the effectiveness of a controller in meeting a specified system requirement. We present an innovative two-layer simulation-based analysis framework designed to identify subtle robustness violations.
  arXiv  Detail & Related papers  (2023-11-13T16:44:43Z)
• Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
  Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations. We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
  arXiv  Detail & Related papers  (2021-06-14T20:48:48Z)
• A new interpretable unsupervised anomaly detection method based on residual explanation [47.187609203210705]
  We present RXP, a new interpretability method to deal with the limitations for AE-based AD in large-scale systems. It stands out for its implementation simplicity, low computational cost and deterministic behavior. In an experiment using data from a real heavy-haul railway line, the proposed method achieved superior performance compared to SHAP.
  arXiv  Detail & Related papers  (2021-03-14T15:35:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.