Constrained Black-Box Attacks Against Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2508.09275v1
• Date: Tue, 12 Aug 2025 18:31:15 GMT
• Title: Constrained Black-Box Attacks Against Multi-Agent Reinforcement Learning
• Authors: Amine Andam, Jamal Bentahar, Mustapha Hedabou,
• Abstract summary: Collaborative multi-agent reinforcement learning (c-MARL) has rapidly evolved, offering state-of-the-art algorithms for real-world applications.<n>However, a key challenge to its widespread adoption is the lack of a thorough investigation into its vulnerabilities to adversarial attacks.<n>This paper investigates new vulnerabilities under more realistic and constrained conditions, assuming an adversary can only collect and perturb the observations of deployed agents.
• Score: 8.080255323094079
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Collaborative multi-agent reinforcement learning (c-MARL) has rapidly evolved, offering state-of-the-art algorithms for real-world applications, including sensitive domains. However, a key challenge to its widespread adoption is the lack of a thorough investigation into its vulnerabilities to adversarial attacks. Existing work predominantly focuses on training-time attacks or unrealistic scenarios, such as access to policy weights or the ability to train surrogate policies. In this paper, we investigate new vulnerabilities under more realistic and constrained conditions, assuming an adversary can only collect and perturb the observations of deployed agents. We also consider scenarios where the adversary has no access at all. We propose simple yet highly effective algorithms for generating adversarial perturbations designed to misalign how victim agents perceive their environment. Our approach is empirically validated on three benchmarks and 22 environments, demonstrating its effectiveness across diverse algorithms and environments. Furthermore, we show that our algorithm is sample-efficient, requiring only 1,000 samples compared to the millions needed by previous methods.

Related papers

• How vulnerable is my policy? Adversarial attacks on modern behavior cloning policies [22.52780232632902]
  This paper presents a comprehensive study of adversarial attacks on Learning from Demonstration (LfD) algorithms.<n>We study the vulnerability of these methods to untargeted, targeted and universal adversarial perturbations.<n>Our experiments on several simulated robotic manipulation tasks reveal that most of the current methods are highly vulnerable to adversarial perturbations.
  arXiv  Detail & Related papers  (2025-02-06T01:17:39Z)
• Multi-granular Adversarial Attacks against Black-box Neural Ranking Models [111.58315434849047]
  We create high-quality adversarial examples by incorporating multi-granular perturbations. We transform the multi-granular attack into a sequential decision-making process. Our attack method surpasses prevailing baselines in both attack effectiveness and imperceptibility.
  arXiv  Detail & Related papers  (2024-04-02T02:08:29Z)
• Adversarial Training Should Be Cast as a Non-Zero-Sum Game [121.95628660889628]
  Two-player zero-sum paradigm of adversarial training has not engendered sufficient levels of robustness. We show that the commonly used surrogate-based relaxation used in adversarial training algorithms voids all guarantees on robustness. A novel non-zero-sum bilevel formulation of adversarial training yields a framework that matches and in some cases outperforms state-of-the-art attacks.
  arXiv  Detail & Related papers  (2023-06-19T16:00:48Z)
• Probabilistic Categorical Adversarial Attack & Adversarial Training [45.458028977108256]
  The existence of adversarial examples brings huge concern for people to apply Deep Neural Networks (DNNs) in safety-critical tasks. How to generate adversarial examples with categorical data is an important problem but lack of extensive exploration. We propose Probabilistic Categorical Adversarial Attack (PCAA), which transfers the discrete optimization problem to a continuous problem that can be solved efficiently by Projected Gradient Descent.
  arXiv  Detail & Related papers  (2022-10-17T19:04:16Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Robust Reinforcement Learning via Genetic Curriculum [5.421464476555662]
  Genetic curriculum is an algorithm that automatically identifies scenarios in which the agent currently fails and generates an associated curriculum. Our empirical studies show improvement in robustness over the existing state of the art algorithms, providing training curricula that result in agents being 2 - 8x times less likely to fail.
  arXiv  Detail & Related papers  (2022-02-17T01:14:20Z)
• Automated Decision-based Adversarial Attacks [48.01183253407982]
  We consider the practical and challenging decision-based black-box adversarial setting. Under this setting, the attacker can only acquire the final classification labels by querying the target model. We propose to automatically discover decision-based adversarial attack algorithms.
  arXiv  Detail & Related papers  (2021-05-09T13:15:10Z)
• MixNet for Generalized Face Presentation Attack Detection [63.35297510471997]
  We have proposed a deep learning-based network termed as textitMixNet to detect presentation attacks. The proposed algorithm utilizes state-of-the-art convolutional neural network architectures and learns the feature mapping for each attack category.
  arXiv  Detail & Related papers  (2020-10-25T23:01:13Z)
• A black-box adversarial attack for poisoning clustering [78.19784577498031]
  We propose a black-box adversarial attack for crafting adversarial samples to test the robustness of clustering algorithms. We show that our attacks are transferable even against supervised algorithms such as SVMs, random forests, and neural networks.
  arXiv  Detail & Related papers  (2020-09-09T18:19:31Z)
• Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey [1.8782750537161614]
  This paper studies strategies to implement adversary robustly trained algorithms towards guaranteeing safety in machine learning algorithms. We provide a taxonomy to classify adversarial attacks and defenses, formulate the Robust Optimization problem in a min-max setting, and divide it into 3 subcategories, namely: Adversarial (re)Training, Regularization Approach, and Certified Defenses.
  arXiv  Detail & Related papers  (2020-07-01T21:00:32Z)
• Protecting Classifiers From Attacks [0.41942958779358663]
  In multiple domains such as malware detection, automated driving systems, or fraud detection, classification algorithms are susceptible to being attacked.<n>We present an alternative Bayesian decision theoretic framework that accounts for the uncertainty about the attacker's behavior.<n>Globally, we are able to robustify statistical classification algorithms against malicious attacks.
  arXiv  Detail & Related papers  (2020-04-18T21:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.