CaT: Balanced Continual Graph Learning with Graph Condensation

• URL: http://arxiv.org/abs/2309.09455v2
• Date: Tue, 19 Sep 2023 01:00:15 GMT
• Title: CaT: Balanced Continual Graph Learning with Graph Condensation
• Authors: Yilun Liu and Ruihong Qiu and Zi Huang
• Abstract summary: Continual graph learning (CGL) is purposed to continuously update a graph model with graph data being fed in a streaming manner. Recent replay-based methods intend to solve this problem by updating the model using both the entire new-coming data and a memory bank that stores replayed graphs. To solve these issues, a Condense and Train framework is proposed in this paper.
• Score: 29.7368211701716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continual graph learning (CGL) is purposed to continuously update a graph model with graph data being fed in a streaming manner. Since the model easily forgets previously learned knowledge when training with new-coming data, the catastrophic forgetting problem has been the major focus in CGL. Recent replay-based methods intend to solve this problem by updating the model using both (1) the entire new-coming data and (2) a sampling-based memory bank that stores replayed graphs to approximate the distribution of historical data. After updating the model, a new replayed graph sampled from the incoming graph will be added to the existing memory bank. Despite these methods are intuitive and effective for the CGL, two issues are identified in this paper. Firstly, most sampling-based methods struggle to fully capture the historical distribution when the storage budget is tight. Secondly, a significant data imbalance exists in terms of the scales of the complex new-coming graph data and the lightweight memory bank, resulting in unbalanced training. To solve these issues, a Condense and Train (CaT) framework is proposed in this paper. Prior to each model update, the new-coming graph is condensed to a small yet informative synthesised replayed graph, which is then stored in a Condensed Graph Memory with historical replay graphs. In the continual learning phase, a Training in Memory scheme is used to update the model directly with the Condensed Graph Memory rather than the whole new-coming graph, which alleviates the data imbalance problem. Extensive experiments conducted on four benchmark datasets successfully demonstrate superior performances of the proposed CaT framework in terms of effectiveness and efficiency. The code has been released on https://github.com/superallen13/CaT-CGL.

Related papers

• E-CGL: An Efficient Continual Graph Learner [30.757231591601997]
  In continual graph learning, graphs evolve based on streaming graph data. Continual graph learning presents unique challenges that require adaptive and efficient graph learning methods. We produce an Efficient Continual Graph Learner (E-CGL) in this paper.
  arXiv  Detail & Related papers  (2024-08-18T04:10:30Z)
• Graph Continual Learning with Debiased Lossless Memory Replay [24.67200419367994]
  Real-life graph data often expands continually, rendering the learning of graph neural networks (GNNs) on static graph data impractical. Graph continual learning (GCL) tackles this problem by continually adapting GNNs to the expanded graph of the current task while maintaining the performance over the graph of previous tasks. Memory replay-based methods, which aim to replay data of previous tasks when learning new tasks, have been explored as one principled approach to mitigate the forgetting of the knowledge learned from the previous tasks. In this paper we extend this methodology with a novel framework, called Debiased Lossless Memory replay (DeLo
  arXiv  Detail & Related papers  (2024-04-17T01:31:00Z)
• Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching [50.30124426442228]
  Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have raised growing concerns. We propose a novel graph method named textbfCraftextbfTing textbfRationatextbf (textbfCTRL) which offers an optimized starting point closer to the original dataset's feature distribution.
  arXiv  Detail & Related papers  (2024-02-07T14:49:10Z)
• A Topology-aware Graph Coarsening Framework for Continual Graph Learning [8.136809136959302]
  Continual learning on graphs tackles the problem of training a graph neural network (GNN) where graph data arrive in a streaming fashion. Traditional continual learning strategies such as Experience Replay can be adapted to streaming graphs. We propose TA$mathbbCO$, a (t)opology-(a)ware graph (co)arsening and (co)ntinual learning framework.
  arXiv  Detail & Related papers  (2024-01-05T22:22:13Z)
• PUMA: Efficient Continual Graph Learning for Node Classification with Graph Condensation [49.00940417190911]
  Existing graph representation learning models encounter a catastrophic problem when learning with newly incoming graphs. In this paper, we propose a PUdo-label guided Memory bAnkrogation (PUMA) framework to enhance its efficiency and effectiveness.
  arXiv  Detail & Related papers  (2023-12-22T05:09:58Z)
• Learning Large Graph Property Prediction via Graph Segment Training [61.344814074335304]
  We propose a general framework that allows learning large graph property prediction with a constant memory footprint. We refine the GST paradigm by introducing a historical embedding table to efficiently obtain embeddings for segments not sampled for backpropagation. Our experiments show that GST-EFD is both memory-efficient and fast, while offering a slight boost on test accuracy over a typical full graph training regime.
  arXiv  Detail & Related papers  (2023-05-21T02:53:25Z)
• Contrastive Graph Few-Shot Learning [67.01464711379187]
  We propose a Contrastive Graph Few-shot Learning framework (CGFL) for graph mining tasks. CGFL learns data representation in a self-supervised manner, thus mitigating the distribution shift impact for better generalization. Comprehensive experiments demonstrate that CGFL outperforms state-of-the-art baselines on several graph mining tasks.
  arXiv  Detail & Related papers  (2022-09-30T20:40:23Z)
• Scaling R-GCN Training with Graph Summarization [71.06855946732296]
  Training of Relation Graph Convolutional Networks (R-GCN) does not scale well with the size of the graph. In this work, we experiment with the use of graph summarization techniques to compress the graph. We obtain reasonable results on the AIFB, MUTAG and AM datasets.
  arXiv  Detail & Related papers  (2022-03-05T00:28:43Z)
• Causal Incremental Graph Convolution for Recommender System Retraining [89.25922726558875]
  Real-world recommender system needs to be regularly retrained to keep with the new data. In this work, we consider how to efficiently retrain graph convolution network (GCN) based recommender models.
  arXiv  Detail & Related papers  (2021-08-16T04:20:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.