The Use of the Simplex Architecture to Enhance Safety in Deep-Learning-Powered Autonomous Systems

• URL: http://arxiv.org/abs/2509.21014v1
• Date: Thu, 25 Sep 2025 11:20:47 GMT
• Title: The Use of the Simplex Architecture to Enhance Safety in Deep-Learning-Powered Autonomous Systems
• Authors: Federico Nesti, Niko Salamini, Mauro Marinoni, Giorgio Maria Cicero, Gabriele Serra, Alessandro Biondi, Giorgio Buttazzo,
• Abstract summary: This paper presents a software architecture for enhancing safety, security, and predictability levels of learning-based autonomous systems.<n>It leverages two isolated execution domains, one dedicated to the execution of neural networks under a rich operating system, which is deemed not trustworthy, and one responsible for running safety-critical functions.
• Score: 38.86794557167231
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Recently, the outstanding performance reached by neural networks in many tasks has led to their deployment in autonomous systems, such as robots and vehicles. However, neural networks are not yet trustworthy, being prone to different types of misbehavior, such as anomalous samples, distribution shifts, adversarial attacks, and other threats. Furthermore, frameworks for accelerating the inference of neural networks typically run on rich operating systems that are less predictable in terms of timing behavior and present larger surfaces for cyber-attacks. To address these issues, this paper presents a software architecture for enhancing safety, security, and predictability levels of learning-based autonomous systems. It leverages two isolated execution domains, one dedicated to the execution of neural networks under a rich operating system, which is deemed not trustworthy, and one responsible for running safety-critical functions, possibly under a different operating system capable of handling real-time constraints. Both domains are hosted on the same computing platform and isolated through a type-1 real-time hypervisor enabling fast and predictable inter-domain communication to exchange real-time data. The two domains cooperate to provide a fail-safe mechanism based on a safety monitor, which oversees the state of the system and switches to a simpler but safer backup module, hosted in the safety-critical domain, whenever its behavior is considered untrustworthy. The effectiveness of the proposed architecture is illustrated by a set of experiments performed on two control systems: a Furuta pendulum and a rover. The results confirm the utility of the fall-back mechanism in preventing faults due to the learning component.

Related papers

• Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs [65.6660735371212]
  We present textbftextscJustAsk, a framework that autonomously discovers effective extraction strategies through interaction alone.<n>It formulates extraction as an online exploration problem, using Upper Confidence Bound--based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration.<n>Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
  arXiv  Detail & Related papers  (2026-01-29T03:53:25Z)
• Securing Agentic AI: Threat Modeling and Risk Analysis for Network Monitoring Agentic AI System [2.5145802129902664]
  The MAESTRO framework was used to expose, evaluate, and eliminate vulnerabilities of agentic AI.<n>The prototype agent system was constructed and implemented, using Python, LangChain, and telemetry in WebSockets.
  arXiv  Detail & Related papers  (2025-08-12T00:14:12Z)
• Building Hybrid B-Spline And Neural Network Operators [0.0]
  Control systems are indispensable for ensuring the safety of cyber-physical systems (CPS) We propose a novel strategy that combines the inductive bias of B-splines with data-driven neural networks to facilitate real-time predictions of CPS behavior.
  arXiv  Detail & Related papers  (2024-06-06T21:54:59Z)
• SCART: Simulation of Cyber Attacks for Real-Time [0.1633272850273525]
  This paper introduces a novel cyber-attack simulation infrastructure designed to enhance simulation environments for real-time systems.<n>We present the SCART framework and dataset, addressing a central challenge in real-time systems: the lack of scalable testing environments.<n>By leveraging simulation-based capabilities, the framework generates training and testing data for data-driven approaches, such as machine learning.
  arXiv  Detail & Related papers  (2023-04-07T14:25:30Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• TESDA: Transform Enabled Statistical Detection of Attacks in Deep Neural Networks [0.0]
  We present TESDA, a low-overhead, flexible, and statistically grounded method for online detection of attacks. Unlike most prior work, we require neither dedicated hardware to run in real-time, nor the presence of a Trojan trigger to detect discrepancies in behavior. We empirically establish our method's usefulness and practicality across multiple architectures, datasets and diverse attacks.
  arXiv  Detail & Related papers  (2021-10-16T02:10:36Z)
• Increasing the Confidence of Deep Neural Networks by Coverage Analysis [71.57324258813674]
  This paper presents a lightweight monitoring architecture based on coverage paradigms to enhance the model against different unsafe inputs. Experimental results show that the proposed approach is effective in detecting both powerful adversarial examples and out-of-distribution inputs.
  arXiv  Detail & Related papers  (2021-01-28T16:38:26Z)
• Scalable Learning of Safety Guarantees for Autonomous Systems using Hamilton-Jacobi Reachability [18.464688553299663]
  Methods like Hamilton-Jacobi reachability can provide guaranteed safe sets and controllers for such systems. As the system is operating, it may learn new knowledge about these uncertainties and should therefore update its safety analysis accordingly. In this paper we synthesize several techniques to speed up computation: decomposition, warm-starting, and adaptive grids.
  arXiv  Detail & Related papers  (2021-01-15T00:13:01Z)
• Dos and Don'ts of Machine Learning in Computer Security [74.1816306998445]
  Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance. We identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible.
  arXiv  Detail & Related papers  (2020-10-19T13:09:31Z)
• Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learning [40.989393438716476]
  Deep Neural Network-based systems are now the state-of-the-art in many robotics tasks, but their application in safety-critical domains remains dangerous without formal guarantees on network robustness. Small perturbations to sensor inputs are often enough to change network-based decisions, which was recently shown to cause an autonomous vehicle to swerve into another lane. This work leverages research on certified adversarial robustness to develop an online certifiably robust for deep reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2020-04-11T21:36:13Z)
• Enhanced Adversarial Strategically-Timed Attacks against Deep Reinforcement Learning [91.13113161754022]
  We introduce timing-based adversarial strategies against a DRL-based navigation system by jamming in physical noise patterns on the selected time frames. Our experimental results show that the adversarial timing attacks can lead to a significant performance drop.
  arXiv  Detail & Related papers  (2020-02-20T21:39:25Z)
• Firearm Detection and Segmentation Using an Ensemble of Semantic Neural Networks [62.997667081978825]
  We present a weapon detection system based on an ensemble of semantic Convolutional Neural Networks. A set of simpler neural networks dedicated to specific tasks requires less computational resources and can be trained in parallel. The overall output of the system given by the aggregation of the outputs of individual networks can be tuned by a user to trade-off false positives and false negatives.
  arXiv  Detail & Related papers  (2020-02-11T13:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.