FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning
- URL: http://arxiv.org/abs/2404.14061v2
- Date: Thu, 25 Apr 2024 06:40:22 GMT
- Title: FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning
- Authors: Yinlin Zhu, Xunkai Li, Zhengyu Wu, Di Wu, Miao Hu, Rong-Hua Li,
- Abstract summary: Subgraph federated learning (subgraph-FL) facilitates the collaborative training of graph neural networks (GNNs) by multi-client subgraphs.
node and topology variation leads to significant differences in the class-wise knowledge reliability of multiple local GNNs.
We propose topology-aware data-free knowledge distillation technology (FedTAD) to enhance reliable knowledge transfer from the local model to the global model.
- Score: 12.834423184614849
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Subgraph federated learning (subgraph-FL) is a new distributed paradigm that facilitates the collaborative training of graph neural networks (GNNs) by multi-client subgraphs. Unfortunately, a significant challenge of subgraph-FL arises from subgraph heterogeneity, which stems from node and topology variation, causing the impaired performance of the global GNN. Despite various studies, they have not yet thoroughly investigated the impact mechanism of subgraph heterogeneity. To this end, we decouple node and topology variation, revealing that they correspond to differences in label distribution and structure homophily. Remarkably, these variations lead to significant differences in the class-wise knowledge reliability of multiple local GNNs, misguiding the model aggregation with varying degrees. Building on this insight, we propose topology-aware data-free knowledge distillation technology (FedTAD), enhancing reliable knowledge transfer from the local model to the global model. Extensive experiments on six public datasets consistently demonstrate the superiority of FedTAD over state-of-the-art baselines.
Related papers
- Federated Learning with Limited Node Labels [3.738399857803053]
Subgraph federated learning (SFL) is a research methodology that has gained significant attention.
We present a novel SFL framework called FedMpa that aims to learn cross-subgraph node representations.
Our experiments on six graph datasets demonstrate that FedMpa is highly effective in node classification.
arXiv Detail & Related papers (2024-06-18T09:30:10Z) - DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment [57.62885438406724]
Graph neural networks are recognized for their strong performance across various applications.
BP has limitations that challenge its biological plausibility and affect the efficiency, scalability and parallelism of training neural networks for graph-based tasks.
We propose DFA-GNN, a novel forward learning framework tailored for GNNs with a case study of semi-supervised learning.
arXiv Detail & Related papers (2024-06-04T07:24:51Z) - Heterophilous Distribution Propagation for Graph Neural Networks [23.897535976924722]
We propose heterophilous distribution propagation (HDP) for graph neural networks.
Instead of aggregating information from all neighborhoods, HDP adaptively separates the neighbors into homophilous and heterphilous parts.
We conduct extensive experiments on 9 benchmark datasets with different levels of homophily.
arXiv Detail & Related papers (2024-05-31T06:40:56Z) - AdaFGL: A New Paradigm for Federated Node Classification with Topology
Heterogeneity [44.11777886421429]
Federated Graph Learning (FGL) has attracted significant attention as a distributed framework based on graph neural networks.
We introduce the concept of structure Non-iid split and then present a new paradigm called underlineAdaptive underlineFederated underlineGraph underlineLearning (AdaFGL)
Our proposed AdaFGL outperforms baselines by significant margins of 3.24% and 5.57% on community split and structure Non-iid split, respectively.
arXiv Detail & Related papers (2024-01-22T08:23:31Z) - Few-Shot Causal Representation Learning for Out-of-Distribution Generalization on Heterogeneous Graphs [16.130356170284127]
Heterogeneous graph few-shot learning (HGFL) has been developed to address the label sparsity issue in heterogeneous graphs (HGs)
We propose a novel Causal OOD Heterogeneous graph Few-shot learning model, namely COHF.
arXiv Detail & Related papers (2024-01-07T22:47:38Z) - MixupExplainer: Generalizing Explanations for Graph Neural Networks with
Data Augmentation [6.307753856507624]
Graph Neural Networks (GNNs) have received increasing attention due to their ability to learn from graph-structured data.
Post-hoc instance-level explanation methods have been proposed to understand GNN predictions.
We shed light on the existence of the distribution shifting issue in existing methods, which affects explanation quality.
arXiv Detail & Related papers (2023-07-15T15:46:38Z) - Learning Strong Graph Neural Networks with Weak Information [64.64996100343602]
We develop a principled approach to the problem of graph learning with weak information (GLWI)
We propose D$2$PT, a dual-channel GNN framework that performs long-range information propagation on the input graph with incomplete structure, but also on a global graph that encodes global semantic similarities.
arXiv Detail & Related papers (2023-05-29T04:51:09Z) - Deep Graph Neural Networks via Flexible Subgraph Aggregation [50.034313206471694]
Graph neural networks (GNNs) can learn from graph-structured data and learn the representation of nodes through aggregating neighborhood information.
In this paper, we evaluate the expressive power of GNNs from the perspective of subgraph aggregation.
We propose a sampling-based node-level residual module (SNR) that can achieve a more flexible utilization of different hops of subgraph aggregation.
arXiv Detail & Related papers (2023-05-09T12:03:42Z) - Position-aware Structure Learning for Graph Topology-imbalance by
Relieving Under-reaching and Over-squashing [67.83086131278904]
Topology-imbalance is a graph-specific imbalance problem caused by the uneven topology positions of labeled nodes.
We propose a novel position-aware graph structure learning framework named PASTEL.
Our key insight is to enhance the connectivity of nodes within the same class for more supervision information.
arXiv Detail & Related papers (2022-08-17T14:04:21Z) - Handling Distribution Shifts on Graphs: An Invariance Perspective [77.14319095965058]
We formulate the OOD problem for node-level prediction on graphs.
We develop a new domain-invariant learning approach, named Explore-to-Extrapolate Risk Minimization.
We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution.
arXiv Detail & Related papers (2022-02-05T02:31:01Z) - Cyclic Label Propagation for Graph Semi-supervised Learning [52.102251202186025]
We introduce a novel framework for graph semi-supervised learning called CycProp.
CycProp integrates GNNs into the process of label propagation in a cyclic and mutually reinforcing manner.
In particular, our proposed CycProp updates the node embeddings learned by GNN module with the augmented information by label propagation.
arXiv Detail & Related papers (2020-11-24T02:55:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.