Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey

• URL: http://arxiv.org/abs/2511.05982v1
• Date: Sat, 08 Nov 2025 12:06:54 GMT
• Title: Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey
• Authors: Albert Schotschneider, Svetlana Pavlitska, J. Marius Zöllner,
• Abstract summary: Deep neural networks (DNNs) are widely used in perception systems for safety-critical applications, such as autonomous driving and robotics.<n>DNNs remain vulnerable to various safety concerns, including generalization errors, out-of-distribution (OOD) inputs, and adversarial attacks.<n>This survey provides a comprehensive overview of runtime safety monitoring approaches.
• Score: 11.690579189801939
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep neural networks (DNNs) are widely used in perception systems for safety-critical applications, such as autonomous driving and robotics. However, DNNs remain vulnerable to various safety concerns, including generalization errors, out-of-distribution (OOD) inputs, and adversarial attacks, which can lead to hazardous failures. This survey provides a comprehensive overview of runtime safety monitoring approaches, which operate in parallel to DNNs during inference to detect these safety concerns without modifying the DNN itself. We categorize existing methods into three main groups: Monitoring inputs, internal representations, and outputs. We analyze the state-of-the-art for each category, identify strengths and limitations, and map methods to the safety concerns they address. In addition, we highlight open challenges and future research directions.

Related papers

• Data Poisoning-based Backdoor Attack Framework against Supervised Learning Rules of Spiking Neural Networks [3.9444202574850755]
  Spiking Neural Networks (SNNs) are known for their low energy consumption and high robustness. This paper explores the robustness performance of SNNs trained by supervised learning rules under backdoor attacks.
  arXiv  Detail & Related papers  (2024-09-24T02:15:19Z)
• Scaling #DNN-Verification Tools with Efficient Bound Propagation and Parallel Computing [57.49021927832259]
  Deep Neural Networks (DNNs) are powerful tools that have shown extraordinary results in many scenarios. However, their intricate designs and lack of transparency raise safety concerns when applied in real-world applications. Formal Verification (FV) of DNNs has emerged as a valuable solution to provide provable guarantees on the safety aspect.
  arXiv  Detail & Related papers  (2023-12-10T13:51:25Z)
• Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees [86.1362094580439]
  We introduce the AllDNN-Verification problem: given a safety property and a DNN, enumerate the set of all the regions of the property input domain which are safe. Due to the #P-hardness of the problem, we propose an efficient approximation method called epsilon-ProVe. Our approach exploits a controllable underestimation of the output reachable sets obtained via statistical prediction of tolerance limits.
  arXiv  Detail & Related papers  (2023-08-18T22:30:35Z)
• Assumption Generation for the Verification of Learning-Enabled Autonomous Systems [7.580719272198119]
  We present an assume-guarantee style compositional approach for the formal verification of system-level safety properties. We illustrate our approach on a case study taken from the autonomous airplanes domain.
  arXiv  Detail & Related papers  (2023-05-27T23:30:27Z)
• The #DNN-Verification Problem: Counting Unsafe Inputs for Deep Neural Networks [94.63547069706459]
  #DNN-Verification problem involves counting the number of input configurations of a DNN that result in a violation of a safety property. We propose a novel approach that returns the exact count of violations. We present experimental results on a set of safety-critical benchmarks.
  arXiv  Detail & Related papers  (2023-01-17T18:32:01Z)
• gRoMA: a Tool for Measuring the Global Robustness of Deep Neural Networks [3.2228025627337864]
  Deep neural networks (DNNs) are at the forefront of cutting-edge technology, and have been achieving remarkable performance in a variety of complex tasks. Their integration into safety-critical systems, such as in the aerospace or automotive domains, poses a significant challenge due to the threat of adversarial inputs. Here, we present gRoMA, an innovative and scalable tool that implements a probabilistic approach to measure the global categorial robustness of a DNN.
  arXiv  Detail & Related papers  (2023-01-05T20:45:23Z)
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety [54.478842696269304]
  The use of deep neural networks (DNNs) in safety-critical applications is challenging due to numerous model-inherent shortcomings. In recent years, a zoo of state-of-the-art techniques aiming to address these safety concerns has emerged. Our paper addresses both machine learning experts and safety engineers.
  arXiv  Detail & Related papers  (2021-04-29T09:54:54Z)
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis [71.57324258813674]
  This paper presents a lightweight monitoring architecture based on coverage paradigms to enhance the model against different unsafe inputs. Experimental results show that the proposed approach is effective in detecting both powerful adversarial examples and out-of-distribution inputs.
  arXiv  Detail & Related papers  (2021-01-28T16:38:26Z)
• Measurement-driven Security Analysis of Imperceptible Impersonation Attacks [54.727945432381716]
  We study the exploitability of Deep Neural Network-based Face Recognition systems. We show that factors such as skin color, gender, and age, impact the ability to carry out an attack on a specific target victim. We also study the feasibility of constructing universal attacks that are robust to different poses or views of the attacker's face.
  arXiv  Detail & Related papers  (2020-08-26T19:27:27Z)
• Supporting DNN Safety Analysis and Retraining through Heatmap-based Unsupervised Learning [1.6414392145248926]
  We propose HUDD, an approach that automatically supports the identification of root causes for DNN errors. HUDD identifies root causes by applying a clustering algorithm to heatmaps capturing the relevance of every DNN neuron on the outcome. Also, HUDD retrains DNNs with images that are automatically selected based on their relatedness to the identified image clusters.
  arXiv  Detail & Related papers  (2020-02-03T16:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.