DPMAC: Differentially Private Communication for Cooperative Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2308.09902v1
• Date: Sat, 19 Aug 2023 04:26:23 GMT
• Title: DPMAC: Differentially Private Communication for Cooperative Multi-Agent Reinforcement Learning
• Authors: Canzhe Zhao, Yanjie Ze, Jing Dong, Baoxiang Wang and Shuai Li
• Abstract summary: Communication lays the foundation for cooperation in human society and in multi-agent reinforcement learning (MARL) We propose the textitdifferentially private multi-agent communication (DPMAC) algorithm, which protects the sensitive information of individual agents by equipping each agent with a local message sender with rigorous $(epsilon, delta)$-differential privacy guarantee. We prove the existence of a Nash equilibrium in cooperative MARL with privacy-preserving communication, which suggests that this problem is game-theoretically learnable.
• Score: 21.961558461211165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Communication lays the foundation for cooperation in human society and in multi-agent reinforcement learning (MARL). Humans also desire to maintain their privacy when communicating with others, yet such privacy concern has not been considered in existing works in MARL. To this end, we propose the \textit{differentially private multi-agent communication} (DPMAC) algorithm, which protects the sensitive information of individual agents by equipping each agent with a local message sender with rigorous $(\epsilon, \delta)$-differential privacy (DP) guarantee. In contrast to directly perturbing the messages with predefined DP noise as commonly done in privacy-preserving scenarios, we adopt a stochastic message sender for each agent respectively and incorporate the DP requirement into the sender, which automatically adjusts the learned message distribution to alleviate the instability caused by DP noise. Further, we prove the existence of a Nash equilibrium in cooperative MARL with privacy-preserving communication, which suggests that this problem is game-theoretically learnable. Extensive experiments demonstrate a clear advantage of DPMAC over baseline methods in privacy-preserving scenarios.

Related papers

• Masked Differential Privacy [64.32494202656801]
  We propose an effective approach called masked differential privacy (DP), which allows for controlling sensitive regions where differential privacy is applied. Our method operates selectively on data and allows for defining non-sensitive-temporal regions without DP application or combining differential privacy with other privacy techniques within data samples.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• The Privacy Power of Correlated Noise in Decentralized Learning [39.48990597191246]
  We propose Decor, a variant of decentralized SGD with differential privacy guarantees. We do so under SecLDP, our new relaxation of local DP, which protects all user communications against an external eavesdropper and curious users.
  arXiv  Detail & Related papers  (2024-05-02T06:14:56Z)
• Differentially Private Reinforcement Learning with Self-Play [18.124829682487558]
  We study the problem of multi-agent reinforcement learning (multi-agent RL) with differential privacy (DP) constraints. We first extend the definitions of Joint DP (JDP) and Local DP (LDP) to two-player zero-sum episodic Markov Games. We design a provably efficient algorithm based on optimistic Nash value and privatization of Bernstein-type bonuses.
  arXiv  Detail & Related papers  (2024-04-11T08:42:51Z)
• Deciphering the Interplay between Local Differential Privacy, Average Bayesian Privacy, and Maximum Bayesian Privacy [5.622065847054885]
  We introduce Bayesian privacy and delve into the relationship between LDP and its Bayesian counterparts, unveiling novel insights into utility-privacy trade-offs. Our work not only lays the groundwork for future empirical exploration but also promises to facilitate the design of privacy-preserving algorithms.
  arXiv  Detail & Related papers  (2024-03-25T10:06:45Z)
• Group Decision-Making among Privacy-Aware Agents [2.4401219403555814]
  Preserving individual privacy and enabling efficient social learning are both important desiderata but seem fundamentally at odds with each other. We do so by controlling information leakage using rigorous statistical guarantees that are based on differential privacy (DP) Our results flesh out the nature of the trade-offs in both cases between the quality of the group decision outcomes, learning accuracy, communication cost, and the level of privacy protections that the agents are afforded.
  arXiv  Detail & Related papers  (2024-02-13T01:38:01Z)
• Breaking the Communication-Privacy-Accuracy Tradeoff with $f$-Differential Privacy [51.11280118806893]
  We consider a federated data analytics problem in which a server coordinates the collaborative data analysis of multiple users with privacy concerns and limited communication capability. We study the local differential privacy guarantees of discrete-valued mechanisms with finite output space through the lens of $f$-differential privacy (DP) More specifically, we advance the existing literature by deriving tight $f$-DP guarantees for a variety of discrete-valued mechanisms.
  arXiv  Detail & Related papers  (2023-02-19T16:58:53Z)
• "Am I Private and If So, how Many?" -- Using Risk Communication Formats for Making Differential Privacy Understandable [0.0]
  We adapt risk communication formats in conjunction with a model for the privacy risks of Differential Privacy. We evaluate these novel privacy communication formats in a crowdsourced study.
  arXiv  Detail & Related papers  (2022-04-08T13:30:07Z)
• Privacy Amplification via Shuffling for Linear Contextual Bandits [51.94904361874446]
  We study the contextual linear bandit problem with differential privacy (DP) We show that it is possible to achieve a privacy/utility trade-off between JDP and LDP by leveraging the shuffle model of privacy. Our result shows that it is possible to obtain a tradeoff between JDP and LDP by leveraging the shuffle model while preserving local privacy.
  arXiv  Detail & Related papers  (2021-12-11T15:23:28Z)
• Private Reinforcement Learning with PAC and Regret Guarantees [69.4202374491817]
  We design privacy preserving exploration policies for episodic reinforcement learning (RL) We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP) We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee.
  arXiv  Detail & Related papers  (2020-09-18T20:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.