Cooperative Multi-Agent Actor-Critic for Privacy-Preserving Load Scheduling in a Residential Microgrid

• URL: http://arxiv.org/abs/2110.02784v1
• Date: Wed, 6 Oct 2021 14:05:26 GMT
• Title: Cooperative Multi-Agent Actor-Critic for Privacy-Preserving Load Scheduling in a Residential Microgrid
• Authors: Zhaoming Qin, Nanqing Dong, Eric P. Xing, Junwei Cao
• Abstract summary: We propose a privacy-preserving multi-agent actor-critic framework where the decentralized actors are trained with distributed critics. The proposed framework can preserve the privacy of the households while simultaneously learn the multi-agent credit assignment mechanism implicitly.
• Score: 71.17179010567123
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a scalable data-driven approach, multi-agent reinforcement learning (MARL) has made remarkable advances in solving the cooperative residential load scheduling problems. However, the common centralized training strategy of MARL algorithms raises privacy risks for involved households. In this work, we propose a privacy-preserving multi-agent actor-critic framework where the decentralized actors are trained with distributed critics, such that both the decentralized execution and the distributed training do not require the global state information. The proposed framework can preserve the privacy of the households while simultaneously learn the multi-agent credit assignment mechanism implicitly. The simulation experiments demonstrate that the proposed framework significantly outperforms the existing privacy-preserving actor-critic framework, and can achieve comparable performance to the state-of-the-art actor-critic framework without privacy constraints.

Related papers

• Decentralized Low-Rank Fine-Tuning of Large Language Models [14.75695352321115]
  We propose Dec-LoRA, a decentralized fine-tuning algorithm for Large Language Models (LLMs) based Low-Rank Adaptation (LoRA) Through experiments on BERT and LLaMA, we demonstrate that Dec-LoRA achieves comparable performance to centralized LoRA under various conditions. These findings highlight the potential of Dec-LoRA for scalable fine-tuning in decentralized environments.
  arXiv  Detail & Related papers  (2025-01-26T01:56:25Z)
• Efficient and Robust Regularized Federated Recommendation [52.24782464815489]
  The recommender system (RSRS) addresses both user preference and privacy concerns. We propose a novel method that incorporates non-uniform gradient descent to improve communication efficiency. RFRecF's superior robustness compared to diverse baselines.
  arXiv  Detail & Related papers  (2024-11-03T12:10:20Z)
• ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization [11.620274237352026]
  offline reinforcement learning (RL) has garnered significant attention for its ability to learn effective policies from pre-collected datasets. MARL presents additional challenges due to the large joint state-action space and the complexity of multi-agent behaviors. We introduce a regularizer in the space of stationary distributions to better handle distributional shift.
  arXiv  Detail & Related papers  (2024-10-02T18:56:10Z)
• Coordination Failure in Cooperative Offline MARL [3.623224034411137]
  We focus on coordination failure and investigate the role of joint actions in multi-agent policy gradients with offline data. By using two-player games as an analytical tool, we demonstrate a simple yet overlooked failure mode of BRUD-based algorithms. We propose an approach to mitigate such failure, by prioritising samples from the dataset based on joint-action similarity.
  arXiv  Detail & Related papers  (2024-07-01T14:51:29Z)
• Noisy Neighbors: Efficient membership inference attacks against LLMs [2.666596421430287]
  This paper introduces an efficient methodology that generates textitnoisy neighbors for a target sample by adding noise in the embedding space. Our findings demonstrate that this approach closely matches the effectiveness of employing shadow models, showing its usability in practical privacy auditing scenarios.
  arXiv  Detail & Related papers  (2024-06-24T12:02:20Z)
• Learning Reward and Policy Jointly from Demonstration and Preference Improves Alignment [58.049113055986375]
  We develop a single stage approach named Alignment with Integrated Human Feedback (AIHF) to train reward models and the policy. The proposed approach admits a suite of efficient algorithms, which can easily reduce to, and leverage, popular alignment algorithms. We demonstrate the efficiency of the proposed solutions with extensive experiments involving alignment problems in LLMs and robotic control problems in MuJoCo.
  arXiv  Detail & Related papers  (2024-06-11T01:20:53Z)
• Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning [32.52811740662061]
  This article introduces DP-LoRA, a novel federated learning algorithm tailored for large language models (LLMs) DP-LoRA preserves data privacy by employing a Gaussian mechanism that adds noise in weight updates, maintaining individual data privacy while facilitating collaborative model training.
  arXiv  Detail & Related papers  (2023-12-29T06:50:38Z)
• Privacy Preserving Multi-Agent Reinforcement Learning in Supply Chains [5.436598805836688]
  This paper addresses privacy concerns in multiagent reinforcement learning (MARL) within the context of supply chains. We propose a game-theoretic, privacy-related mechanism, utilizing a secure multi-party computation framework in MARL settings. We present a learning mechanism that carries out floating point operations in a privacy-preserving manner.
  arXiv  Detail & Related papers  (2023-12-09T21:25:21Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond [57.10914865054868]
  We consider vertical logistic regression (VLR) trained with mini-batch descent gradient. We provide a comprehensive and rigorous privacy analysis of VLR in a class of open-source Federated Learning frameworks.
  arXiv  Detail & Related papers  (2022-07-19T05:47:30Z)
• RACA: Relation-Aware Credit Assignment for Ad-Hoc Cooperation in Multi-Agent Deep Reinforcement Learning [55.55009081609396]
  We propose a novel method, called Relation-Aware Credit Assignment (RACA), which achieves zero-shot generalization in ad-hoc cooperation scenarios. RACA takes advantage of a graph-based encoder relation to encode the topological structure between agents. Our method outperforms baseline methods on the StarCraftII micromanagement benchmark and ad-hoc cooperation scenarios.
  arXiv  Detail & Related papers  (2022-06-02T03:39:27Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Decentralized Stochastic Optimization with Inherent Privacy Protection [103.62463469366557]
  Decentralized optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data, privacy protection has become an increasingly pressing need in the implementation of decentralized optimization algorithms.
  arXiv  Detail & Related papers  (2022-05-08T14:38:23Z)
• Secure Distributed/Federated Learning: Prediction-Privacy Trade-Off for Multi-Agent System [4.190359509901197]
  In the big data era, performing inference within the distributed and federated learning (DL and FL) frameworks, the central server needs to process a large amount of data. Considering the decentralized computing topology, privacy has become a first-class concern. We study the textitprivacy-aware server to multi-agent assignment problem subject to information processing constraints associated with each agent.
  arXiv  Detail & Related papers  (2022-04-24T19:19:20Z)
• MA-Dreamer: Coordination and communication through shared imagination [5.253168177256072]
  We present MA-Dreamer, a model-based method that uses both agent-centric and global differentiable models of the environment. Our experiments show that in long-term speaker-listener tasks and in cooperative games with strong partial-observability, MA-Dreamer finds a solution that makes effective use of coordination.
  arXiv  Detail & Related papers  (2022-04-10T13:54:26Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Hybrid Dynamic Contrast and Probability Distillation for Unsupervised Person Re-Id [109.1730454118532]
  Unsupervised person re-identification (Re-Id) has attracted increasing attention due to its practical application in the read-world video surveillance system. We present the hybrid dynamic cluster contrast and probability distillation algorithm. It formulates the unsupervised Re-Id problem into an unified local-to-global dynamic contrastive learning and self-supervised probability distillation framework.
  arXiv  Detail & Related papers  (2021-09-29T02:56:45Z)
• A Privacy-Preserving and Trustable Multi-agent Learning Framework [34.28936739262812]
  This paper presents Privacy-preserving and trustable Distributed Learning (PT-DL) PT-DL is a fully decentralized framework that relies on Differential Privacy to guarantee strong privacy protections of the agents' data. The paper shows that PT-DL is resilient up to a 50% collusion attack, with high probability, in a malicious trust model.
  arXiv  Detail & Related papers  (2021-06-02T15:46:27Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.