Constraints Satisfiability Driven Reinforcement Learning for Autonomous Cyber Defense

• URL: http://arxiv.org/abs/2104.08994v1
• Date: Mon, 19 Apr 2021 01:08:30 GMT
• Title: Constraints Satisfiability Driven Reinforcement Learning for Autonomous Cyber Defense
• Authors: Ashutosh Dutta, Ehab Al-Shaer, and Samrat Chatterjee
• Abstract summary: We present a new hybrid autonomous agent architecture that aims to optimize and verify defense policies of reinforcement learning (RL) We use constraints verification (using satisfiability modulo theory (SMT)) to steer the RL decision-making toward safe and effective actions. Our evaluation of the presented approach in a simulated CPS environment shows that the agent learns the optimal policy fast and defeats diversified attack strategies in 99% cases.
• Score: 7.321728608775741
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing system complexity and attack sophistication, the necessity of autonomous cyber defense becomes vivid for cyber and cyber-physical systems (CPSs). Many existing frameworks in the current state-of-the-art either rely on static models with unrealistic assumptions, or fail to satisfy the system safety and security requirements. In this paper, we present a new hybrid autonomous agent architecture that aims to optimize and verify defense policies of reinforcement learning (RL) by incorporating constraints verification (using satisfiability modulo theory (SMT)) into the agent's decision loop. The incorporation of SMT does not only ensure the satisfiability of safety and security requirements, but also provides constant feedback to steer the RL decision-making toward safe and effective actions. This approach is critically needed for CPSs that exhibit high risk due to safety or security violations. Our evaluation of the presented approach in a simulated CPS environment shows that the agent learns the optimal policy fast and defeats diversified attack strategies in 99\% cases.

Related papers

• A Complexity-Informed Approach to Optimise Cyber Defences [0.0]
  This paper introduces a novel complexity-informed approach to cybersecurity management, addressing the challenges found within complex cyber defences. We adapt and extend the complexity theory to cybersecurity and develop a quantitative framework that empowers decision-makers with strategies to de-complexify defences, identify improvement opportunities, and resolve bottlenecks.
  arXiv  Detail & Related papers  (2025-01-26T16:04:13Z)
• ACTISM: Threat-informed Dynamic Security Modelling for Automotive Systems [7.3347982474177185]
  ACTISM (Automotive Consequence-Driven and Threat-Informed Security Modelling) is an integrated security modelling framework. It enhances the resilience of automotive systems by dynamically updating their cybersecurity posture. We demonstrate the effectiveness of ACTISM by applying it to a real-world example of the Tesla Electric Vehicle's In-Vehicle Infotainment system. We report the results of a practitioners' survey on the usefulness of ACTISM and its future directions.
  arXiv  Detail & Related papers  (2024-11-30T09:58:48Z)
• ActSafe: Active Exploration with Safety Constraints for Reinforcement Learning [48.536695794883826]
  We present ActSafe, a novel model-based RL algorithm for safe and efficient exploration. We show that ActSafe guarantees safety during learning while also obtaining a near-optimal policy in finite time. In addition, we propose a practical variant of ActSafe that builds on latest model-based RL advancements.
  arXiv  Detail & Related papers  (2024-10-12T10:46:02Z)
• Threat-Informed Cyber Resilience Index: A Probabilistic Quantitative Approach to Measure Defence Effectiveness Against Cyber Attacks [0.36832029288386137]
  This paper introduces the Cyber Resilience Index (CRI), a threat-informed probabilistic approach to quantifying an organisation's defence effectiveness against cyber-attacks (campaigns) Building upon the Threat-Intelligence Based Security Assessment (TIBSA) methodology, we present a mathematical model that translates complex threat intelligence into an actionable, unified metric similar to a stock market index, that executives can understand and interact with while teams can act upon.
  arXiv  Detail & Related papers  (2024-06-27T17:51:48Z)
• Modular Control Architecture for Safe Marine Navigation: Reinforcement Learning and Predictive Safety Filters [0.0]
  Reinforcement learning is increasingly used to adapt to complex scenarios, but standard frameworks ensuring safety and stability are lacking. Predictive Safety Filters (PSF) offer a promising solution, ensuring constraint satisfaction in learning-based control without explicit constraint handling. We apply this approach to marine navigation, combining RL with PSF on a simulated Cybership II model. Results demonstrate the PSF's effectiveness in maintaining safety without hindering the RL agent's learning rate and performance, evaluated against a standard RL agent without PSF.
  arXiv  Detail & Related papers  (2023-12-04T12:37:54Z)
• Approximate Model-Based Shielding for Safe Reinforcement Learning [83.55437924143615]
  We propose a principled look-ahead shielding algorithm for verifying the performance of learned RL policies. Our algorithm differs from other shielding approaches in that it does not require prior knowledge of the safety-relevant dynamics of the system. We demonstrate superior performance to other safety-aware approaches on a set of Atari games with state-dependent safety-labels.
  arXiv  Detail & Related papers  (2023-07-27T15:19:45Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks [70.76757529955577]
  This paper revisits prior work in this scope from the perspective of state-wise safe RL. We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection. To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
  arXiv  Detail & Related papers  (2022-12-12T06:30:17Z)
• Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions [60.26921219698514]
  We introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers. We then present the pointwise feasibility conditions of the resulting safety controller. We use these conditions to devise an event-triggered online data collection strategy.
  arXiv  Detail & Related papers  (2022-08-23T05:02:09Z)
• Safe Reinforcement Learning via Confidence-Based Filters [78.39359694273575]
  We develop a control-theoretic approach for certifying state safety constraints for nominal policies learned via standard reinforcement learning techniques. We provide formal safety guarantees, and empirically demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2022-07-04T11:43:23Z)
• Reinforcement Learning for Feedback-Enabled Cyber Resilience [24.92055101652206]
  Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism ( CRM) adapts to the known or zero-day threats and uncertainties in real-time. We review the literature on RL for cyber resiliency and discuss the cyber-resilient defenses against three major types of vulnerabilities.
  arXiv  Detail & Related papers  (2021-07-02T01:08:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.