BRNES: Enabling Security and Privacy-aware Experience Sharing in Multiagent Robotic and Autonomous Systems

• URL: http://arxiv.org/abs/2308.01274v1
• Date: Wed, 2 Aug 2023 16:57:19 GMT
• Title: BRNES: Enabling Security and Privacy-aware Experience Sharing in Multiagent Robotic and Autonomous Systems
• Authors: Md Tamjid Hossain, Hung Manh La, Shahriar Badsha, and Anton Netchaev
• Abstract summary: We propose a novel MARL framework (BRNES) that selects a dynamic neighbor zone for each advisee at each learning step. Our experiments show that our framework outperforms the state-of-the-art in terms of the steps to goal, obtained reward, and time to goal metrics.
• Score: 0.15749416770494704
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Although experience sharing (ES) accelerates multiagent reinforcement learning (MARL) in an advisor-advisee framework, attempts to apply ES to decentralized multiagent systems have so far relied on trusted environments and overlooked the possibility of adversarial manipulation and inference. Nevertheless, in a real-world setting, some Byzantine attackers, disguised as advisors, may provide false advice to the advisee and catastrophically degrade the overall learning performance. Also, an inference attacker, disguised as an advisee, may conduct several queries to infer the advisors' private information and make the entire ES process questionable in terms of privacy leakage. To address and tackle these issues, we propose a novel MARL framework (BRNES) that heuristically selects a dynamic neighbor zone for each advisee at each learning step and adopts a weighted experience aggregation technique to reduce Byzantine attack impact. Furthermore, to keep the agent's private information safe from adversarial inference attacks, we leverage the local differential privacy (LDP)-induced noise during the ES process. Our experiments show that our framework outperforms the state-of-the-art in terms of the steps to goal, obtained reward, and time to goal metrics. Particularly, our evaluation shows that the proposed framework is 8.32x faster than the current non-private frameworks and 1.41x faster than the private frameworks in an adversarial setting.

Related papers

• Secure Aggregation is Not Private Against Membership Inference Attacks [66.59892736942953]
  We investigate the privacy implications of SecAgg in federated learning. We show that SecAgg offers weak privacy against membership inference attacks even in a single training round. Our findings underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection.
  arXiv  Detail & Related papers  (2024-03-26T15:07:58Z)
• Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms [79.61176746380718]
  Multi-Agent Reinforcement Learning (MARL) has shown promising results across several domains. MARL policies often lack robustness and are sensitive to small changes in their environment. We show that we can gain robustness by controlling a policy's Lipschitz constant. We propose a new robust MARL framework, ERNIE, that promotes the Lipschitz continuity of the policies.
  arXiv  Detail & Related papers  (2023-10-16T20:14:06Z)
• Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy [6.343100139647636]
  Federated learning (FL) is designed to preserve data privacy during model training. FL is vulnerable to privacy attacks and Byzantine attacks. We propose a new FL scheme that guarantees rigorous privacy and simultaneously enhances system robustness against Byzantine attacks.
  arXiv  Detail & Related papers  (2023-09-07T01:39:02Z)
• Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond [57.10914865054868]
  We consider vertical logistic regression (VLR) trained with mini-batch descent gradient. We provide a comprehensive and rigorous privacy analysis of VLR in a class of open-source Federated Learning frameworks.
  arXiv  Detail & Related papers  (2022-07-19T05:47:30Z)
• Multi-Agent Constrained Policy Optimisation [17.772811770726296]
  We formulate the safe MARL problem as a constrained Markov game and solve it with policy optimisation methods. Our solutions -- Multi-Agent Constrained Policy optimisation (MACPO) and MAPPO-Lagrangian -- leverage the theories from both constrained policy optimisation and multi-agent trust region learning. We develop the benchmark suite of Safe Multi-Agent MuJoCo that involves a variety of MARL baselines.
  arXiv  Detail & Related papers  (2021-10-06T14:17:09Z)
• Cooperative Multi-Agent Actor-Critic for Privacy-Preserving Load Scheduling in a Residential Microgrid [71.17179010567123]
  We propose a privacy-preserving multi-agent actor-critic framework where the decentralized actors are trained with distributed critics. The proposed framework can preserve the privacy of the households while simultaneously learn the multi-agent credit assignment mechanism implicitly.
  arXiv  Detail & Related papers  (2021-10-06T14:05:26Z)
• Robust Reinforcement Learning on State Observations with Learned Optimal Adversary [86.0846119254031]
  We study the robustness of reinforcement learning with adversarially perturbed state observations. With a fixed agent policy, we demonstrate that an optimal adversary to perturb state observations can be found. For DRL settings, this leads to a novel empirical adversarial attack to RL agents via a learned adversary that is much stronger than previous ones.
  arXiv  Detail & Related papers  (2021-01-21T05:38:52Z)
• Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients [54.98496284653234]
  We consider the task of training a policy that maximizes reward while minimizing disclosure of certain sensitive state variables through the actions. We solve this problem by introducing a regularizer based on the mutual information between the sensitive state and the actions. We develop a model-based estimator for optimization of privacy-constrained policies.
  arXiv  Detail & Related papers  (2020-12-30T03:22:35Z)
• Federated Recommendation System via Differential Privacy [31.0963615274522]
  We explore how differential privacy based Upper Confidence Bound (UCB) methods can be applied to multi-agent environments. We provide a theoretical analysis on the privacy and regret performance of the proposed methods.
  arXiv  Detail & Related papers  (2020-05-14T00:00:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.