Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning

• URL: http://arxiv.org/abs/2108.03803v1
• Date: Mon, 9 Aug 2021 04:41:47 GMT
• Title: Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning
• Authors: Wanqi Xue, Wei Qiu, Bo An, Zinovi Rabinovich, Svetlana Obraztsova, Chai Kiat Yeo
• Abstract summary: We make the first step towards conducting message attacks on MACRL methods. We develop a defence method via message reconstruction. We consider the ability of the malicious agent to adapt to the changing and improving defensive communicative policies.
• Score: 37.24674549469648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies in multi-agent communicative reinforcement learning (MACRL) demonstrate that multi-agent coordination can be significantly improved when communication between agents is allowed. Meanwhile, advances in adversarial machine learning (ML) have shown that ML and reinforcement learning (RL) models are vulnerable to a variety of attacks that significantly degrade the performance of learned behaviours. However, despite the obvious and growing importance, the combination of adversarial ML and MACRL remains largely uninvestigated. In this paper, we make the first step towards conducting message attacks on MACRL methods. In our formulation, one agent in the cooperating group is taken over by an adversary and can send malicious messages to disrupt a deployed MACRL-based coordinated strategy during the deployment phase. We further our study by developing a defence method via message reconstruction. Finally, we address the resulting arms race, i.e., we consider the ability of the malicious agent to adapt to the changing and improving defensive communicative policies of the benign agents. Specifically, we model the adversarial MACRL problem as a two-player zero-sum game and then utilize Policy-Space Response Oracle to achieve communication robustness. Empirically, we demonstrate that MACRL methods are vulnerable to message attacks while our defence method the game-theoretic framework can effectively improve the robustness of MACRL.

Related papers

• Efficient Adversarial Training in LLMs with Continuous Attacks [99.5882845458567]
  Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses. C-AdvIPO is an adversarial variant of IPO that does not require utility data for adversarially robust alignment.
  arXiv  Detail & Related papers  (2024-05-24T14:20:09Z)
• Robust Communicative Multi-Agent Reinforcement Learning with Active Defense [38.6815513394882]
  We propose an active defense strategy, where agents automatically reduce the impact of potentially harmful messages on the final decision. We design an Active Defense Multi-Agent Communication framework (ADMAC), which estimates the reliability of received messages and adjusts their impact on the final decision. The superiority of ADMAC over existing methods is validated by experiments in three communication-critical tasks under four types of attacks.
  arXiv  Detail & Related papers  (2023-12-16T09:02:56Z)
• Enhancing the Robustness of QMIX against State-adversarial Attacks [6.627954554805906]
  We discuss four techniques to improve the robustness of SARL algorithms and extend them to multi-agent scenarios. We train models using a variety of attacks in this research. We then test the models taught using the other attacks by subjecting them to the corresponding attacks throughout the training phase.
  arXiv  Detail & Related papers  (2023-07-03T10:10:34Z)
• Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems [51.6210785955659]
  Communication is important in many multi-agent reinforcement learning (MARL) problems for agents to share information and make good decisions. However, when deploying trained communicative agents in a real-world application where noise and potential attackers exist, the safety of communication-based policies becomes a severe issue that is underexplored. In this work, we consider an environment with $N$ agents, where the attacker may arbitrarily change the communication from any $CfracN-12$ agents to a victim agent.
  arXiv  Detail & Related papers  (2022-06-21T07:32:18Z)
• Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning [20.401609420707867]
  We propose a distributed formation and obstacle avoidance method based on multi-agent reinforcement learning (MARL) Our method achieves better performance regarding formation error, formation convergence rate and on-par success rate of obstacle avoidance compared with baselines.
  arXiv  Detail & Related papers  (2021-11-14T13:02:45Z)
• Adaptive Adversarial Training for Meta Reinforcement Learning [6.576665763018747]
  We build upon model-agnostic meta-learning (MAML) and propose a novel method to generate adversarial samples for MRL by using Generative Adversarial Network (GAN) That allows us to enhance the robustness of MRL to adversal attacks by leveraging these attacks during meta training process.
  arXiv  Detail & Related papers  (2021-04-27T16:23:34Z)
• Robust Reinforcement Learning on State Observations with Learned Optimal Adversary [86.0846119254031]
  We study the robustness of reinforcement learning with adversarially perturbed state observations. With a fixed agent policy, we demonstrate that an optimal adversary to perturb state observations can be found. For DRL settings, this leads to a novel empirical adversarial attack to RL agents via a learned adversary that is much stronger than previous ones.
  arXiv  Detail & Related papers  (2021-01-21T05:38:52Z)
• Adversarial Attacks On Multi-Agent Communication [80.4392160849506]
  Modern autonomous systems will soon be deployed at scale, opening up the possibility for cooperative multi-agent systems. Such advantages rely heavily on communication channels which have been shown to be vulnerable to security breaches. In this paper, we explore such adversarial attacks in a novel multi-agent setting where agents communicate by sharing learned intermediate representations.
  arXiv  Detail & Related papers  (2021-01-17T00:35:26Z)
• Robust Deep Reinforcement Learning through Adversarial Loss [74.20501663956604]
  Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs. We propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against adversarial attacks.
  arXiv  Detail & Related papers  (2020-08-05T07:49:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.