Deep Reinforcement Learning for Autonomous Cyber Operations: A Survey
- URL: http://arxiv.org/abs/2310.07745v1
- Date: Wed, 11 Oct 2023 16:24:14 GMT
- Title: Deep Reinforcement Learning for Autonomous Cyber Operations: A Survey
- Authors: Gregory Palmer, Chris Parry, Daniel J.B. Harrold, Chris Willis
- Abstract summary: The rapid increase in the number of cyber-attacks in recent years raises the need for principled methods for defending networks against malicious actors.
Deep reinforcement learning has emerged as a promising approach for mitigating these attacks.
While DRL has shown much potential for cyber-defence, numerous challenges must be overcome before DRL can be applied to autonomous cyber-operations at scale.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: The rapid increase in the number of cyber-attacks in recent years raises the
need for principled methods for defending networks against malicious actors.
Deep reinforcement learning (DRL) has emerged as a promising approach for
mitigating these attacks. However, while DRL has shown much potential for
cyber-defence, numerous challenges must be overcome before DRL can be applied
to autonomous cyber-operations (ACO) at scale. Principled methods are required
for environments that confront learners with very high-dimensional state
spaces, large multi-discrete action spaces, and adversarial learning. Recent
works have reported success in solving these problems individually. There have
also been impressive engineering efforts towards solving all three for
real-time strategy games. However, applying DRL to the full ACO problem remains
an open challenge. Here, we survey the relevant DRL literature and
conceptualize an idealised ACO-DRL agent. We provide: i.) A summary of the
domain properties that define the ACO problem; ii.) A comprehensive evaluation
of the extent to which domains used for benchmarking DRL approaches are
comparable to ACO; iii.) An overview of state-of-the-art approaches for scaling
DRL to domains that confront learners with the curse of dimensionality, and;
iv.) A survey and critique of current methods for limiting the exploitability
of agents within adversarial settings from the perspective of ACO. We conclude
with open research questions that we hope will motivate future directions for
researchers and practitioners working on ACO.
Related papers
- ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for
Last-Iterate Convergence in Constrained MDPs [31.663072540757643]
Reinforcement Learning (RL) has been applied to real-world problems with increasing success.
We introduce Reinforcement Learning with Optimistic Ascent-Descent (ReLOAD)
arXiv Detail & Related papers (2023-02-02T18:05:27Z) - A Survey of Meta-Reinforcement Learning [83.95180398234238]
We cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL.
We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task.
We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.
arXiv Detail & Related papers (2023-01-19T12:01:41Z) - Tutorial on Course-of-Action (COA) Attack Search Methods in Computer
Networks [8.78504593920219]
As the network size grows, the traditional course-of-action (COA) attack search methods can suffer from the limitations to computing and communication resources.
reinforcement learning (RL)-based intelligent algorithms are one of the most effective solutions.
arXiv Detail & Related papers (2022-05-27T05:37:07Z) - A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open
Problems [0.0]
Reinforcement learning (RL) has experienced a dramatic increase in popularity.
There is still a wide range of domains inaccessible to RL due to the high cost and danger of interacting with the environment.
offline RL is a paradigm that learns exclusively from static datasets of previously collected interactions.
arXiv Detail & Related papers (2022-03-02T20:05:11Z) - URLB: Unsupervised Reinforcement Learning Benchmark [82.36060735454647]
We introduce the Unsupervised Reinforcement Learning Benchmark (URLB)
URLB consists of two phases: reward-free pre-training and downstream task adaptation with extrinsic rewards.
We provide twelve continuous control tasks from three domains for evaluation and open-source code for eight leading unsupervised RL methods.
arXiv Detail & Related papers (2021-10-28T15:07:01Z) - Robust Deep Reinforcement Learning through Adversarial Loss [74.20501663956604]
Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs.
We propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against adversarial attacks.
arXiv Detail & Related papers (2020-08-05T07:49:42Z) - What Matters In On-Policy Reinforcement Learning? A Large-Scale
Empirical Study [50.79125250286453]
On-policy reinforcement learning (RL) has been successfully applied to many different continuous control tasks.
But state-of-the-art implementations take numerous low- and high-level design decisions that strongly affect the performance of the resulting agents.
These choices are usually not extensively discussed in the literature, leading to discrepancy between published descriptions of algorithms and their implementations.
We implement >50 such choices'' in a unified on-policy RL framework, allowing us to investigate their impact in a large-scale empirical study.
arXiv Detail & Related papers (2020-06-10T17:59:03Z) - Robust Deep Reinforcement Learning against Adversarial Perturbations on
State Observations [88.94162416324505]
A deep reinforcement learning (DRL) agent observes its states through observations, which may contain natural measurement errors or adversarial noises.
Since the observations deviate from the true states, they can mislead the agent into making suboptimal actions.
We show that naively applying existing techniques on improving robustness for classification tasks, like adversarial training, is ineffective for many RL tasks.
arXiv Detail & Related papers (2020-03-19T17:59:59Z) - Challenges and Countermeasures for Adversarial Attacks on Deep
Reinforcement Learning [48.49658986576776]
Deep Reinforcement Learning (DRL) has numerous applications in the real world thanks to its outstanding ability in adapting to the surrounding environments.
Despite its great advantages, DRL is susceptible to adversarial attacks, which precludes its use in real-life critical systems and applications.
This paper presents emerging attacks in DRL-based systems and the potential countermeasures to defend against these attacks.
arXiv Detail & Related papers (2020-01-27T10:53:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.