Towards Heterogeneous Continual Graph Learning via Meta-knowledge Distillation

• URL: http://arxiv.org/abs/2505.17458v1
• Date: Fri, 23 May 2025 04:37:57 GMT
• Title: Towards Heterogeneous Continual Graph Learning via Meta-knowledge Distillation
• Authors: Guiquan Sun, Xikun Zhang, Jingchao Ni, Dongjin Song,
• Abstract summary: This work addresses the challenge of continual learning on heterogeneous graphs by introducing the Meta-learning based Knowledge Distillation framework (MKD)<n>MKD combines rapid task adaptation through meta-learning on limited samples with knowledge distillation to achieve an optimal balance between incorporating new information and maintaining existing knowledge.<n> Comprehensive evaluations across three benchmark datasets validate MKD's effectiveness in handling continual learning scenarios on expanding heterogeneous graphs.
• Score: 21.129688460913485
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning on heterogeneous graphs has experienced rapid advancement in recent years, driven by the inherently heterogeneous nature of real-world data. However, existing studies typically assume the graphs to be static, while real-world graphs are continuously expanding. This dynamic nature requires models to adapt to new data while preserving existing knowledge. To this end, this work addresses the challenge of continual learning on heterogeneous graphs by introducing the Meta-learning based Knowledge Distillation framework (MKD), designed to mitigate catastrophic forgetting in evolving heterogeneous graph structures. MKD combines rapid task adaptation through meta-learning on limited samples with knowledge distillation to achieve an optimal balance between incorporating new information and maintaining existing knowledge. To improve the efficiency and effectiveness of sample selection, MKD incorporates a novel sampling strategy that selects a small number of target-type nodes based on node diversity and maintains fixed-size buffers for other types. The strategy retrieves first-order neighbors along metapaths and selects important neighbors based on their structural relevance, enabling the sampled subgraphs to retain key topological and semantic information. In addition, MKD introduces a semantic-level distillation module that aligns the attention distributions over different metapaths between teacher and student models, encouraging semantic consistency beyond the logit level. Comprehensive evaluations across three benchmark datasets validate MKD's effectiveness in handling continual learning scenarios on expanding heterogeneous graphs.

Related papers

• Adversarial Curriculum Graph-Free Knowledge Distillation for Graph Neural Networks [61.608453110751206]
  We propose a fast and high-quality data-free knowledge distillation approach for graph neural networks.<n>The proposed graph-free KD method (ACGKD) significantly reduces the spatial complexity of pseudo-graphs.<n>ACGKD eliminates the dimensional ambiguity between the student and teacher models by increasing the student's dimensions.
  arXiv  Detail & Related papers  (2025-04-01T08:44:27Z)
• A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptation [59.14165404728197]
  We provide an up-to-date and forward-looking review of deep graph learning under distribution shifts.<n>Specifically, we cover three primary scenarios: graph OOD generalization, training-time graph OOD adaptation, and test-time graph OOD adaptation.<n>To provide a better understanding of the literature, we introduce a systematic taxonomy that classifies existing methods into model-centric and data-centric approaches.
  arXiv  Detail & Related papers  (2024-10-25T02:39:56Z)
• LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphs [22.322402072526927]
  Heterogeneous Graph Contrastive Learning (HGCL) usually requires pre-defined meta-paths. textsfLAMP integrates various meta-path sub-graphs into a unified and stable structure. textsfLAMP significantly outperforms existing state-of-the-art unsupervised models in terms of accuracy and robustness.
  arXiv  Detail & Related papers  (2024-09-10T08:27:39Z)
• M2HGCL: Multi-Scale Meta-Path Integrated Heterogeneous Graph Contrastive Learning [16.391439666603578]
  We propose a new multi-scale meta-path integrated heterogeneous graph contrastive learning (M2HGCL) model. Specifically, we expand the meta-paths and jointly aggregate the direct neighbor information, the initial meta-path neighbor information and the expanded meta-path neighbor information. Through extensive experiments on three real-world datasets, we demonstrate that M2HGCL outperforms the current state-of-the-art baseline models.
  arXiv  Detail & Related papers  (2023-09-03T06:39:56Z)
• Epistemic Graph: A Plug-And-Play Module For Hybrid Representation Learning [46.48026220464475]
  Humans exhibit hybrid learning, seamlessly integrating structured knowledge for cross-domain recognition or relying on a smaller amount of data samples for few-shot learning. We introduce a novel Epistemic Graph Layer (EGLayer) to enable hybrid learning, enhancing the exchange of information between deep features and a structured knowledge graph.
  arXiv  Detail & Related papers  (2023-05-30T04:10:15Z)
• Data-heterogeneity-aware Mixing for Decentralized Learning [63.83913592085953]
  We characterize the dependence of convergence on the relationship between the mixing weights of the graph and the data heterogeneity across nodes. We propose a metric that quantifies the ability of a graph to mix the current gradients. Motivated by our analysis, we propose an approach that periodically and efficiently optimize the metric.
  arXiv  Detail & Related papers  (2022-04-13T15:54:35Z)
• Meta Propagation Networks for Graph Few-shot Semi-supervised Learning [39.96930762034581]
  We propose a novel network architecture equipped with a novel meta-learning algorithm to solve this problem. In essence, our framework Meta-PN infers high-quality pseudo labels on unlabeled nodes via a meta-learned label propagation strategy. Our approach offers easy and substantial performance gains compared to existing techniques on various benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-18T00:11:56Z)
• Weakly-supervised Graph Meta-learning for Few-shot Node Classification [53.36828125138149]
  We propose a new graph meta-learning framework -- Graph Hallucination Networks (Meta-GHN) Based on a new robustness-enhanced episodic training, Meta-GHN is meta-learned to hallucinate clean node representations from weakly-labeled data. Extensive experiments demonstrate the superiority of Meta-GHN over existing graph meta-learning studies.
  arXiv  Detail & Related papers  (2021-06-12T22:22:10Z)
• Structure-Enhanced Meta-Learning For Few-Shot Graph Classification [53.54066611743269]
  This work explores the potential of metric-based meta-learning for solving few-shot graph classification. An implementation upon GIN, named SMFGIN, is tested on two datasets, Chembl and TRIANGLES.
  arXiv  Detail & Related papers  (2021-03-05T09:03:03Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.