FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning

• URL: http://arxiv.org/abs/2404.14061v2
• Date: Thu, 25 Apr 2024 06:40:22 GMT
• Title: FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning
• Authors: Yinlin Zhu, Xunkai Li, Zhengyu Wu, Di Wu, Miao Hu, Rong-Hua Li,
• Abstract summary: Subgraph federated learning (subgraph-FL) facilitates the collaborative training of graph neural networks (GNNs) by multi-client subgraphs. node and topology variation leads to significant differences in the class-wise knowledge reliability of multiple local GNNs. We propose topology-aware data-free knowledge distillation technology (FedTAD) to enhance reliable knowledge transfer from the local model to the global model.
• Score: 12.834423184614849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Subgraph federated learning (subgraph-FL) is a new distributed paradigm that facilitates the collaborative training of graph neural networks (GNNs) by multi-client subgraphs. Unfortunately, a significant challenge of subgraph-FL arises from subgraph heterogeneity, which stems from node and topology variation, causing the impaired performance of the global GNN. Despite various studies, they have not yet thoroughly investigated the impact mechanism of subgraph heterogeneity. To this end, we decouple node and topology variation, revealing that they correspond to differences in label distribution and structure homophily. Remarkably, these variations lead to significant differences in the class-wise knowledge reliability of multiple local GNNs, misguiding the model aggregation with varying degrees. Building on this insight, we propose topology-aware data-free knowledge distillation technology (FedTAD), enhancing reliable knowledge transfer from the local model to the global model. Extensive experiments on six public datasets consistently demonstrate the superiority of FedTAD over state-of-the-art baselines.

Related papers

• DeCaf: A Causal Decoupling Framework for OOD Generalization on Node Classification [14.96980804513399]
  Graph Neural Networks (GNNs) are susceptible to distribution shifts, creating vulnerability and security issues in critical domains. Existing methods that target learning an invariant (feature, structure)-label mapping often depend on oversimplified assumptions about the data generation process. We introduce a more realistic graph data generation model using Structural Causal Models (SCMs) We propose a casual decoupling framework, DeCaf, that independently learns unbiased feature-label and structure-label mappings.
  arXiv  Detail & Related papers  (2024-10-27T00:22:18Z)
• The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challenges [101.83124435649358]
  Homophily principle, ie nodes with the same labels or similar attributes are more likely to be connected. Recent work has identified a non-trivial set of datasets where GNN's performance compared to the NN's is not satisfactory.
  arXiv  Detail & Related papers  (2024-07-12T18:04:32Z)
• Federated Learning with Limited Node Labels [3.738399857803053]
  Subgraph federated learning (SFL) is a research methodology that has gained significant attention. We present a novel SFL framework called FedMpa that aims to learn cross-subgraph node representations. Our experiments on six graph datasets demonstrate that FedMpa is highly effective in node classification.
  arXiv  Detail & Related papers  (2024-06-18T09:30:10Z)
• DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment [57.62885438406724]
  Graph neural networks are recognized for their strong performance across various applications. BP has limitations that challenge its biological plausibility and affect the efficiency, scalability and parallelism of training neural networks for graph-based tasks. We propose DFA-GNN, a novel forward learning framework tailored for GNNs with a case study of semi-supervised learning.
  arXiv  Detail & Related papers  (2024-06-04T07:24:51Z)
• Heterophilous Distribution Propagation for Graph Neural Networks [23.897535976924722]
  We propose heterophilous distribution propagation (HDP) for graph neural networks. Instead of aggregating information from all neighborhoods, HDP adaptively separates the neighbors into homophilous and heterphilous parts. We conduct extensive experiments on 9 benchmark datasets with different levels of homophily.
  arXiv  Detail & Related papers  (2024-05-31T06:40:56Z)
• AdaFGL: A New Paradigm for Federated Node Classification with Topology Heterogeneity [44.11777886421429]
  Federated Graph Learning (FGL) has attracted significant attention as a distributed framework based on graph neural networks. We introduce the concept of structure Non-iid split and then present a new paradigm called underlineAdaptive underlineFederated underlineGraph underlineLearning (AdaFGL) Our proposed AdaFGL outperforms baselines by significant margins of 3.24% and 5.57% on community split and structure Non-iid split, respectively.
  arXiv  Detail & Related papers  (2024-01-22T08:23:31Z)
• Few-Shot Causal Representation Learning for Out-of-Distribution Generalization on Heterogeneous Graphs [16.130356170284127]
  Heterogeneous graph few-shot learning (HGFL) has been developed to address the label sparsity issue in heterogeneous graphs (HGs) We propose a novel Causal OOD Heterogeneous graph Few-shot learning model, namely COHF.
  arXiv  Detail & Related papers  (2024-01-07T22:47:38Z)
• Learning Strong Graph Neural Networks with Weak Information [64.64996100343602]
  We develop a principled approach to the problem of graph learning with weak information (GLWI) We propose D$2$PT, a dual-channel GNN framework that performs long-range information propagation on the input graph with incomplete structure, but also on a global graph that encodes global semantic similarities.
  arXiv  Detail & Related papers  (2023-05-29T04:51:09Z)
• Position-aware Structure Learning for Graph Topology-imbalance by Relieving Under-reaching and Over-squashing [67.83086131278904]
  Topology-imbalance is a graph-specific imbalance problem caused by the uneven topology positions of labeled nodes. We propose a novel position-aware graph structure learning framework named PASTEL. Our key insight is to enhance the connectivity of nodes within the same class for more supervision information.
  arXiv  Detail & Related papers  (2022-08-17T14:04:21Z)
• Handling Distribution Shifts on Graphs: An Invariance Perspective [78.31180235269035]
  We formulate the OOD problem on graphs and develop a new invariant learning approach, Explore-to-Extrapolate Risk Minimization (EERM) EERM resorts to multiple context explorers that are adversarially trained to maximize the variance of risks from multiple virtual environments. We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution.
  arXiv  Detail & Related papers  (2022-02-05T02:31:01Z)
• Cyclic Label Propagation for Graph Semi-supervised Learning [52.102251202186025]
  We introduce a novel framework for graph semi-supervised learning called CycProp. CycProp integrates GNNs into the process of label propagation in a cyclic and mutually reinforcing manner. In particular, our proposed CycProp updates the node embeddings learned by GNN module with the augmented information by label propagation.
  arXiv  Detail & Related papers  (2020-11-24T02:55:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.