Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey

• URL: http://arxiv.org/abs/2403.00420v1
• Date: Fri, 1 Mar 2024 10:16:46 GMT
• Title: Robust Deep Reinforcement Learning Through Adversarial Attacks and Training : A Survey
• Authors: Lucas Schott, Josephine Delas, Hatem Hajri, Elies Gherbi, Reda Yaich, Nora Boulahia-Cuppens, Frederic Cuppens, Sylvain Lamprier
• Abstract summary: Deep Reinforcement Learning (DRL) is an approach for training autonomous agents across various complex environments. It remains susceptible to minor conditions variations, raising concerns about its reliability in real-world applications. A way to improve robustness of DRL to unknown changes in the conditions is through Adversarial Training.
• Score: 8.463282079069362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep Reinforcement Learning (DRL) is an approach for training autonomous agents across various complex environments. Despite its significant performance in well known environments, it remains susceptible to minor conditions variations, raising concerns about its reliability in real-world applications. To improve usability, DRL must demonstrate trustworthiness and robustness. A way to improve robustness of DRL to unknown changes in the conditions is through Adversarial Training, by training the agent against well suited adversarial attacks on the dynamics of the environment. Addressing this critical issue, our work presents an in-depth analysis of contemporary adversarial attack methodologies, systematically categorizing them and comparing their objectives and operational mechanisms. This classification offers a detailed insight into how adversarial attacks effectively act for evaluating the resilience of DRL agents, thereby paving the way for enhancing their robustness.

Related papers

• Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantization [18.56608399174564]
  Well-performing reinforcement learning (RL) agents often lack resilience against adversarial perturbations during deployment. This highlights the importance of building a robust agent before deploying it in the real world. In this work, we study an input transformation-based defense for RL.
  arXiv  Detail & Related papers  (2024-10-04T12:41:54Z)
• Robust Image Classification: Defensive Strategies against FGSM and PGD Adversarial Attacks [0.0]
  Adversarial attacks pose significant threats to the robustness of deep learning models in image classification. This paper explores and refines defense mechanisms against these attacks to enhance the resilience of neural networks.
  arXiv  Detail & Related papers  (2024-08-20T02:00:02Z)
• Towards Robust Policy: Enhancing Offline Reinforcement Learning with Adversarial Attacks and Defenses [19.918548094276005]
  offline reinforcement learning (RL) addresses the challenge of expensive and high-risk data exploration inherent in RL by pre-training policies on vast amounts of offline data. This paper proposes a framework to enhance the robustness of offline RL models by leveraging advanced adversarial attacks and defenses.
  arXiv  Detail & Related papers  (2024-05-18T07:23:44Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• Improve Robustness of Reinforcement Learning against Observation Perturbations via $l_\infty$ Lipschitz Policy Networks [8.39061976254379]
  Deep Reinforcement Learning (DRL) has achieved remarkable advances in sequential decision tasks. Recent works have revealed that DRL agents are susceptible to slight perturbations in observations. We propose a novel robust reinforcement learning method called SortRL, which improves the robustness of DRL policies against observation perturbations.
  arXiv  Detail & Related papers  (2023-12-14T08:57:22Z)
• Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation [78.17108227614928]
  We propose a benchmark environment for Safe Reinforcement Learning focusing on aquatic navigation. We consider a value-based and policy-gradient Deep Reinforcement Learning (DRL) We also propose a verification strategy that checks the behavior of the trained models over a set of desired properties.
  arXiv  Detail & Related papers  (2021-12-16T16:53:56Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)
• Policy Smoothing for Provably Robust Reinforcement Learning [109.90239627115336]
  We study the provable robustness of reinforcement learning against norm-bounded adversarial perturbations of the inputs. We generate certificates that guarantee that the total reward obtained by the smoothed policy will not fall below a certain threshold under a norm-bounded adversarial of perturbation the input.
  arXiv  Detail & Related papers  (2021-06-21T21:42:08Z)
• Robust Reinforcement Learning on State Observations with Learned Optimal Adversary [86.0846119254031]
  We study the robustness of reinforcement learning with adversarially perturbed state observations. With a fixed agent policy, we demonstrate that an optimal adversary to perturb state observations can be found. For DRL settings, this leads to a novel empirical adversarial attack to RL agents via a learned adversary that is much stronger than previous ones.
  arXiv  Detail & Related papers  (2021-01-21T05:38:52Z)
• Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations [88.94162416324505]
  A deep reinforcement learning (DRL) agent observes its states through observations, which may contain natural measurement errors or adversarial noises. Since the observations deviate from the true states, they can mislead the agent into making suboptimal actions. We show that naively applying existing techniques on improving robustness for classification tasks, like adversarial training, is ineffective for many RL tasks.
  arXiv  Detail & Related papers  (2020-03-19T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.