PPSGCN: A Privacy-Preserving Subgraph Sampling Based Distributed GCN
Training Method
- URL: http://arxiv.org/abs/2110.12906v1
- Date: Fri, 22 Oct 2021 08:22:36 GMT
- Title: PPSGCN: A Privacy-Preserving Subgraph Sampling Based Distributed GCN
Training Method
- Authors: Binchi Zhang, Minnan Luo, Shangbin Feng, Ziqi Liu, Jun Zhou, Qinghua
Zheng
- Abstract summary: Graph convolutional networks (GCNs) have been widely adopted for graph representation learning and achieved impressive performance.
Existing methods directly exchange node features between different clients, which results in data privacy leakage.
We propose a Privacy-Preserving Subgraph sampling based distributed GCN training method (PPSGCN) which preserves data privacy and significantly cuts back on communication and memory overhead.
- Score: 28.829761038950707
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Graph convolutional networks (GCNs) have been widely adopted for graph
representation learning and achieved impressive performance. For larger graphs
stored separately on different clients, distributed GCN training algorithms
were proposed to improve efficiency and scalability. However, existing methods
directly exchange node features between different clients, which results in
data privacy leakage. Federated learning was incorporated in graph learning to
tackle data privacy, while they suffer from severe performance drop due to
non-iid data distribution. Besides, these approaches generally involve heavy
communication and memory overhead during the training process. In light of
these problems, we propose a Privacy-Preserving Subgraph sampling based
distributed GCN training method (PPSGCN), which preserves data privacy and
significantly cuts back on communication and memory overhead. Specifically,
PPSGCN employs a star-topology client-server system. We firstly sample a local
node subset in each client to form a global subgraph, which greatly reduces
communication and memory costs. We then conduct local computation on each
client with features or gradients of the sampled nodes. Finally, all clients
securely communicate with the central server with homomorphic encryption to
combine local results while preserving data privacy. Compared with federated
graph learning methods, our PPSGCN model is trained on a global graph to avoid
the negative impact of local data distribution. We prove that our PPSGCN
algorithm would converge to a local optimum with probability 1. Experiment
results on three prevalent benchmarks demonstrate that our algorithm
significantly reduces communication and memory overhead while maintaining
desirable performance. Further studies not only demonstrate the fast
convergence of PPSGCN, but discuss the trade-off between communication and
local computation cost as well.
Related papers
- Federated Face Forgery Detection Learning with Personalized Representation [63.90408023506508]
Deep generator technology can produce high-quality fake videos that are indistinguishable, posing a serious social threat.
Traditional forgery detection methods directly centralized training on data.
The paper proposes a novel federated face forgery detection learning with personalized representation.
arXiv Detail & Related papers (2024-06-17T02:20:30Z) - FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation [7.052566906745796]
FedLPA is a layer-wise posterior aggregation method for federated learning.
We show that FedLPA significantly improves learning performance over state-of-the-art methods across several metrics.
arXiv Detail & Related papers (2023-09-30T10:51:27Z) - FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup
for Non-IID Data [54.81695390763957]
Federated learning is an emerging distributed machine learning method.
We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate.
We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
arXiv Detail & Related papers (2023-09-18T12:35:05Z) - Distributed Learning over Networks with Graph-Attention-Based
Personalization [49.90052709285814]
We propose a graph-based personalized algorithm (GATTA) for distributed deep learning.
In particular, the personalized model in each agent is composed of a global part and a node-specific part.
By treating each agent as one node in a graph the node-specific parameters as its features, the benefits of the graph attention mechanism can be inherited.
arXiv Detail & Related papers (2023-05-22T13:48:30Z) - Acceleration of Federated Learning with Alleviated Forgetting in Local
Training [61.231021417674235]
Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy.
We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage.
Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
arXiv Detail & Related papers (2022-03-05T02:31:32Z) - FedGCN: Convergence-Communication Tradeoffs in Federated Training of
Graph Convolutional Networks [14.824579000821272]
We introduce the Federated Graph Convolutional Network (FedGCN) algorithm, which uses federated learning to train GCN models for semi-supervised node classification.
Compared to prior methods that require extra communication among clients at each training round, FedGCN clients only communicate with the central server in one pre-training step.
Experimental results show that our FedGCN algorithm achieves better model accuracy with 51.7% faster convergence on average and at least 100X less communication compared to prior work.
arXiv Detail & Related papers (2022-01-28T21:39:16Z) - Local Augmentation for Graph Neural Networks [78.48812244668017]
We introduce the local augmentation, which enhances node features by its local subgraph structures.
Based on the local augmentation, we further design a novel framework: LA-GNN, which can apply to any GNN models in a plug-and-play manner.
arXiv Detail & Related papers (2021-09-08T18:10:08Z) - Exploiting Shared Representations for Personalized Federated Learning [54.65133770989836]
We propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client.
Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation.
This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions.
arXiv Detail & Related papers (2021-02-14T05:36:25Z) - Communication-Efficient Sampling for Distributed Training of Graph
Convolutional Networks [3.075766050800645]
Training Graph Convolutional Networks (GCNs) is expensive as it needs to aggregate data from neighboring nodes.
Previous works have proposed various neighbor sampling methods that estimate the aggregation result based on a small number of sampled neighbors.
We present an algorithm that determines the local sampling probabilities and makes sure our skewed neighbor sampling does not affect much the convergence of the training.
arXiv Detail & Related papers (2021-01-19T16:12:44Z) - Privacy Amplification by Decentralization [0.0]
We introduce a novel relaxation of local differential privacy (LDP) that naturally arises in fully decentralized protocols.
We study a decentralized model of computation where a token performs a walk on the network graph and is updated sequentially by the party who receives it.
We prove that the privacy-utility trade-offs of our algorithms significantly improve upon LDP, and in some cases even match what can be achieved with methods based on trusted/secure aggregation and shuffling.
arXiv Detail & Related papers (2020-12-09T21:33:33Z) - Locally Private Graph Neural Networks [12.473486843211573]
We study the problem of node data privacy, where graph nodes have potentially sensitive data that is kept private.
We develop a privacy-preserving, architecture-agnostic GNN learning algorithm with formal privacy guarantees.
Experiments conducted over real-world datasets demonstrate that our method can maintain a satisfying level of accuracy with low privacy loss.
arXiv Detail & Related papers (2020-06-09T22:36:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.