FedGAT: A Privacy-Preserving Federated Approximation Algorithm for Graph Attention Networks

• URL: http://arxiv.org/abs/2412.16144v1
• Date: Fri, 20 Dec 2024 18:48:46 GMT
• Title: FedGAT: A Privacy-Preserving Federated Approximation Algorithm for Graph Attention Networks
• Authors: Siddharth Ambekar, Yuhang Yao, Ryan Li, Carlee Joe-Wong,
• Abstract summary: We introduce the Federated Graph Attention Network (FedGAT) algorithm for semi-supervised node classification. FedGAT achieves nearly the same accuracy as a GAT model in a centralised setting, and its performance is robust to the number of clients as well as data distribution.
• Score: 14.791684379921598
• License:
• Abstract: Federated training methods have gained popularity for graph learning with applications including friendship graphs of social media sites and customer-merchant interaction graphs of huge online marketplaces. However, privacy regulations often require locally generated data to be stored on local clients. The graph is then naturally partitioned across clients, with no client permitted access to information stored on another. Cross-client edges arise naturally in such cases and present an interesting challenge to federated training methods, as training a graph model at one client requires feature information of nodes on the other end of cross-client edges. Attempting to retain such edges often incurs significant communication overhead, and dropping them altogether reduces model performance. In simpler models such as Graph Convolutional Networks, this can be fixed by communicating a limited amount of feature information across clients before training, but GATs (Graph Attention Networks) require additional information that cannot be pre-communicated, as it changes from training round to round. We introduce the Federated Graph Attention Network (FedGAT) algorithm for semi-supervised node classification, which approximates the behavior of GATs with provable bounds on the approximation error. FedGAT requires only one pre-training communication round, significantly reducing the communication overhead for federated GAT training. We then analyze the error in the approximation and examine the communication overhead and computational complexity of the algorithm. Experiments show that FedGAT achieves nearly the same accuracy as a GAT model in a centralised setting, and its performance is robust to the number of clients as well as data distribution.

Related papers

• Communication-Efficient Personalized Federal Graph Learning via Low-Rank Decomposition [18.99572321624751]
  We propose a communication-efficient personalized graph learning algorithm, CEFGL. Our method decomposes the model parameters into low-rank generic and sparse private models. We employ a dual-channel encoder to learn sparse local knowledge in a personalized manner.
  arXiv  Detail & Related papers  (2024-12-18T02:26:07Z)
• Federated Graph Learning with Graphless Clients [52.5629887481768]
  Federated Graph Learning (FGL) is tasked with training machine learning models, such as Graph Neural Networks (GNNs) We propose a novel framework FedGLS to tackle the problem in FGL with graphless clients.
  arXiv  Detail & Related papers  (2024-11-13T06:54:05Z)
• Distributed Training of Large Graph Neural Networks with Variable Communication Rates [71.7293735221656]
  Training Graph Neural Networks (GNNs) on large graphs presents unique challenges due to the large memory and computing requirements. Distributed GNN training, where the graph is partitioned across multiple machines, is a common approach to training GNNs on large graphs. We introduce a variable compression scheme for reducing the communication volume in distributed GNN training without compromising the accuracy of the learned model.
  arXiv  Detail & Related papers  (2024-06-25T14:57:38Z)
• Distributed Learning over Networks with Graph-Attention-Based Personalization [49.90052709285814]
  We propose a graph-based personalized algorithm (GATTA) for distributed deep learning. In particular, the personalized model in each agent is composed of a global part and a node-specific part. By treating each agent as one node in a graph the node-specific parameters as its features, the benefits of the graph attention mechanism can be inherited.
  arXiv  Detail & Related papers  (2023-05-22T13:48:30Z)
• Learnable Graph Matching: A Practical Paradigm for Data Association [74.28753343714858]
  We propose a general learnable graph matching method to address these issues. Our method achieves state-of-the-art performance on several MOT datasets. For image matching, our method outperforms state-of-the-art methods on a popular indoor dataset, ScanNet.
  arXiv  Detail & Related papers  (2023-03-27T17:39:00Z)
• Graph Federated Learning with Hidden Representation Sharing [33.01999333117515]
  Learning on Graphs (LoG) is widely used in multi-client systems when each client has insufficient local data. Federated Learning (FL) requires models to be trained in a multi-client system and sharing of raw data among clients. In this work, we first formulate the Graph Federated Learning (GFL) problem that unifies LoG FL in multi-client systems and then propose sharing hidden representation.
  arXiv  Detail & Related papers  (2022-12-23T05:44:27Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks [14.824579000821272]
  We introduce the Federated Graph Convolutional Network (FedGCN) algorithm, which uses federated learning to train GCN models for semi-supervised node classification. Compared to prior methods that require extra communication among clients at each training round, FedGCN clients only communicate with the central server in one pre-training step. Experimental results show that our FedGCN algorithm achieves better model accuracy with 51.7% faster convergence on average and at least 100X less communication compared to prior work.
  arXiv  Detail & Related papers  (2022-01-28T21:39:16Z)
• FedGraph: Federated Graph Learning with Intelligent Sampling [7.798227884125872]
  Federated learning has attracted much research attention due to its privacy protection in distributed machine learning. Existing work of federated learning mainly focuses on Convolutional Neural Network (CNN), which cannot efficiently handle graph data that are popular in many applications. In this paper, we propose FedGraph for federated graph learning among multiple computing clients, each of which holds a subgraph.
  arXiv  Detail & Related papers  (2021-11-02T04:58:03Z)
• Training Robust Graph Neural Networks with Topology Adaptive Edge Dropping [116.26579152942162]
  Graph neural networks (GNNs) are processing architectures that exploit graph structural information to model representations from network data. Despite their success, GNNs suffer from sub-optimal generalization performance given limited training data. This paper proposes Topology Adaptive Edge Dropping to improve generalization performance and learn robust GNN models.
  arXiv  Detail & Related papers  (2021-06-05T13:20:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.