PP-MARL: Efficient Privacy-Preserving Multi-Agent Reinforcement Learning for Cooperative Intelligence in Communications

• URL: http://arxiv.org/abs/2204.12064v2
• Date: Fri, 21 Feb 2025 16:23:57 GMT
• Title: PP-MARL: Efficient Privacy-Preserving Multi-Agent Reinforcement Learning for Cooperative Intelligence in Communications
• Authors: Tingting Yuan, Hwei-Ming Chung, Xiaoming Fu,
• Abstract summary: Multi-agent reinforcement learning (MARL) is a popular approach for achieving cooperative intelligence (CI) in communication problems.<n> Ensuring privacy protection for MARL is a challenging task because of the presence of heterogeneous agents that learn interdependently via sharing information.<n>We propose PP-MARL, an efficient privacy-preserving learning scheme for MARL.
• Score: 15.955599283219298
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cooperative intelligence (CI) is expected to become an integral element in next-generation networks because it can aggregate the capabilities and intelligence of multiple devices. Multi-agent reinforcement learning (MARL) is a popular approach for achieving CI in communication problems by enabling effective collaboration among agents to address sequential problems. However, ensuring privacy protection for MARL is a challenging task because of the presence of heterogeneous agents that learn interdependently via sharing information. Implementing privacy protection techniques such as data encryption and federated learning to MARL introduces the notable overheads (e.g., computation and bandwidth). To overcome these challenges, we propose PP-MARL, an efficient privacy-preserving learning scheme for MARL. PP-MARL leverages homomorphic encryption (HE) and differential privacy (DP) to protect privacy, while introducing split learning to decrease overheads via reducing the volume of shared messages, and then improve efficiency. We apply and evaluate PP-MARL in two communication-related use cases. Simulation results reveal that PP-MARL can achieve efficient and reliable collaboration with 1.1-6 times better privacy protection and lower overheads (e.g., 84-91% reduction in bandwidth) than state-of-the-art approaches.

Related papers

• Privacy-Enhancing Paradigms within Federated Multi-Agent Systems [47.76990892943637]
  LLM-based Multi-Agent Systems (MAS) have proven highly effective in solving complex problems by integrating multiple agents, each performing different roles. In this paper, we introduce the concept of Federated MAS, highlighting the fundamental differences between Federated MAS and traditional FL. We then identify key challenges in developing Federated MAS, including: 1) heterogeneous privacy protocols among agents, 2) structural differences in multi-party conversations, and 3) dynamic conversational network structures. To address these challenges, we propose Embedded Privacy-Enhancing Agents (EPEAgent), an innovative solution that integrates seamlessly into the Retrieval-Augmented Generation phase and the
  arXiv  Detail & Related papers  (2025-03-11T08:38:45Z)
• Collaborative Inference over Wireless Channels with Feature Differential Privacy [57.68286389879283]
  Collaborative inference among multiple wireless edge devices has the potential to significantly enhance Artificial Intelligence (AI) applications. transmitting extracted features poses a significant privacy risk, as sensitive personal data can be exposed during the process. We propose a novel privacy-preserving collaborative inference mechanism, wherein each edge device in the network secures the privacy of extracted features before transmitting them to a central server for inference.
  arXiv  Detail & Related papers  (2024-10-25T18:11:02Z)
• Enhancing Physical Layer Communication Security through Generative AI with Mixture of Experts [80.0638227807621]
  generative artificial intelligence (GAI) models have demonstrated superiority over conventional AI methods. MoE, which uses multiple expert models for prediction through a gate mechanism, proposes possible solutions.
  arXiv  Detail & Related papers  (2024-05-07T11:13:17Z)
• Beyond Joint Demonstrations: Personalized Expert Guidance for Efficient Multi-Agent Reinforcement Learning [54.40927310957792]
  We introduce a novel concept of personalized expert demonstrations, tailored for each individual agent or, more broadly, each individual type of agent within a heterogeneous team. These demonstrations solely pertain to single-agent behaviors and how each agent can achieve personal goals without encompassing any cooperative elements. We propose an approach that selectively utilizes personalized expert demonstrations as guidance and allows agents to learn to cooperate.
  arXiv  Detail & Related papers  (2024-03-13T20:11:20Z)
• Decentralized and Lifelong-Adaptive Multi-Agent Collaborative Learning [57.652899266553035]
  Decentralized and lifelong-adaptive multi-agent collaborative learning aims to enhance collaboration among multiple agents without a central server. We propose DeLAMA, a decentralized multi-agent lifelong collaborative learning algorithm with dynamic collaboration graphs.
  arXiv  Detail & Related papers  (2024-03-11T09:21:11Z)
• The Security and Privacy of Mobile Edge Computing: An Artificial Intelligence Perspective [64.36680481458868]
  Mobile Edge Computing (MEC) is a new computing paradigm that enables cloud computing and information technology (IT) services to be delivered at the network's edge. This paper provides a survey of security and privacy in MEC from the perspective of Artificial Intelligence (AI) We focus on new security and privacy issues, as well as potential solutions from the viewpoints of AI.
  arXiv  Detail & Related papers  (2024-01-03T07:47:22Z)
• Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning [32.52811740662061]
  This article introduces DP-LoRA, a novel federated learning algorithm tailored for large language models (LLMs) DP-LoRA preserves data privacy by employing a Gaussian mechanism that adds noise in weight updates, maintaining individual data privacy while facilitating collaborative model training.
  arXiv  Detail & Related papers  (2023-12-29T06:50:38Z)
• Privacy Preserving Multi-Agent Reinforcement Learning in Supply Chains [5.436598805836688]
  This paper addresses privacy concerns in multiagent reinforcement learning (MARL) within the context of supply chains. We propose a game-theoretic, privacy-related mechanism, utilizing a secure multi-party computation framework in MARL settings. We present a learning mechanism that carries out floating point operations in a privacy-preserving manner.
  arXiv  Detail & Related papers  (2023-12-09T21:25:21Z)
• AlberDICE: Addressing Out-Of-Distribution Joint Actions in Offline Multi-Agent RL via Alternating Stationary Distribution Correction Estimation [65.4532392602682]
  One of the main challenges in offline Reinforcement Learning (RL) is the distribution shift that arises from the learned policy deviating from the data collection policy. This is often addressed by avoiding out-of-distribution (OOD) actions during policy improvement as their presence can lead to substantial performance degradation. We introduce AlberDICE, an offline MARL algorithm that performs centralized training of individual agents based on stationary distribution optimization.
  arXiv  Detail & Related papers  (2023-11-03T18:56:48Z)
• DPMAC: Differentially Private Communication for Cooperative Multi-Agent Reinforcement Learning [21.961558461211165]
  Communication lays the foundation for cooperation in human society and in multi-agent reinforcement learning (MARL) We propose the textitdifferentially private multi-agent communication (DPMAC) algorithm, which protects the sensitive information of individual agents by equipping each agent with a local message sender with rigorous $(epsilon, delta)$-differential privacy guarantee. We prove the existence of a Nash equilibrium in cooperative MARL with privacy-preserving communication, which suggests that this problem is game-theoretically learnable.
  arXiv  Detail & Related papers  (2023-08-19T04:26:23Z)
• Federated Learning-Empowered AI-Generated Content in Wireless Networks [58.48381827268331]
  Federated learning (FL) can be leveraged to improve learning efficiency and achieve privacy protection for AIGC. We present FL-based techniques for empowering AIGC, and aim to enable users to generate diverse, personalized, and high-quality content.
  arXiv  Detail & Related papers  (2023-07-14T04:13:11Z)
• Building Cooperative Embodied Agents Modularly with Large Language Models [104.57849816689559]
  We address challenging multi-agent cooperation problems with decentralized control, raw sensory observations, costly communication, and multi-objective tasks instantiated in various embodied environments. We harness the commonsense knowledge, reasoning ability, language comprehension, and text generation prowess of LLMs and seamlessly incorporate them into a cognitive-inspired modular framework. Our experiments on C-WAH and TDW-MAT demonstrate that CoELA driven by GPT-4 can surpass strong planning-based methods and exhibit emergent effective communication.
  arXiv  Detail & Related papers  (2023-07-05T17:59:27Z)
• Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning [18.824571167583432]
  Communication efficiency and privacy protection are critical issues in distributed machine learning. We propose a comprehensive quantization-based solution that could simultaneously achieve communication efficiency and privacy protection. We theoretically capture the new trade-offs between communication, privacy, and learning performance.
  arXiv  Detail & Related papers  (2023-04-26T13:13:04Z)
• Privacy-Preserving Joint Edge Association and Power Optimization for the Internet of Vehicles via Federated Multi-Agent Reinforcement Learning [74.53077322713548]
  We investigate the privacy-preserving joint edge association and power allocation problem. The proposed solution strikes a compelling trade-off, while preserving a higher privacy level than the state-of-the-art solutions.
  arXiv  Detail & Related papers  (2023-01-26T10:09:23Z)
• Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond [57.10914865054868]
  We consider vertical logistic regression (VLR) trained with mini-batch descent gradient. We provide a comprehensive and rigorous privacy analysis of VLR in a class of open-source Federated Learning frameworks.
  arXiv  Detail & Related papers  (2022-07-19T05:47:30Z)
• Privacy-Preserving Communication-Efficient Federated Multi-Armed Bandits [17.039484057126337]
  Communication bottleneck and data privacy are two critical concerns in federated multi-armed bandit (MAB) problems. We design the privacy-preserving communication-efficient algorithm in such problems and study the interactions among privacy, communication and learning performance in terms of the regret.
  arXiv  Detail & Related papers  (2021-11-02T12:56:12Z)
• Learning Individually Inferred Communication for Multi-Agent Cooperation [37.56115000150748]
  We propose Individually Inferred Communication (I2C) to enable agents to learn a prior for agent-agent communication. The prior knowledge is learned via causal inference and realized by a feed-forward neural network. I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios.
  arXiv  Detail & Related papers  (2020-06-11T14:07:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.