Continual Learning on Dynamic Graphs via Parameter Isolation
- URL: http://arxiv.org/abs/2305.13825v2
- Date: Tue, 11 Jul 2023 08:02:42 GMT
- Title: Continual Learning on Dynamic Graphs via Parameter Isolation
- Authors: Peiyan Zhang, Yuchen Yan, Chaozhuo Li, Senzhang Wang, Xing Xie, Guojie
Song, Sunghun Kim
- Abstract summary: We propose Isolation GNN (PI-GNN) for continual learning on dynamic graphs.
We find parameters that correspond to unaffected patterns via optimization and freeze them to prevent them from being rewritten.
Experiments on eight real-world datasets corroborate the effectiveness of PI-GNN.
- Score: 40.96053483180836
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Many real-world graph learning tasks require handling dynamic graphs where
new nodes and edges emerge. Dynamic graph learning methods commonly suffer from
the catastrophic forgetting problem, where knowledge learned for previous
graphs is overwritten by updates for new graphs. To alleviate the problem,
continual graph learning methods are proposed. However, existing continual
graph learning methods aim to learn new patterns and maintain old ones with the
same set of parameters of fixed size, and thus face a fundamental tradeoff
between both goals. In this paper, we propose Parameter Isolation GNN (PI-GNN)
for continual learning on dynamic graphs that circumvents the tradeoff via
parameter isolation and expansion. Our motivation lies in that different
parameters contribute to learning different graph patterns. Based on the idea,
we expand model parameters to continually learn emerging graph patterns.
Meanwhile, to effectively preserve knowledge for unaffected patterns, we find
parameters that correspond to them via optimization and freeze them to prevent
them from being rewritten. Experiments on eight real-world datasets corroborate
the effectiveness of PI-GNN compared to state-of-the-art baselines.
Related papers
- Gradient Transformation: Towards Efficient and Model-Agnostic Unlearning for Dynamic Graph Neural Networks [66.70786325911124]
Graph unlearning has emerged as an essential tool for safeguarding user privacy and mitigating the negative impacts of undesirable data.
With the increasing prevalence of DGNNs, it becomes imperative to investigate the implementation of dynamic graph unlearning.
We propose an effective, efficient, model-agnostic, and post-processing method to implement DGNN unlearning.
arXiv Detail & Related papers (2024-05-23T10:26:18Z) - A Topology-aware Graph Coarsening Framework for Continual Graph Learning [8.136809136959302]
Continual learning on graphs tackles the problem of training a graph neural network (GNN) where graph data arrive in a streaming fashion.
Traditional continual learning strategies such as Experience Replay can be adapted to streaming graphs.
We propose TA$mathbbCO$, a (t)opology-(a)ware graph (co)arsening and (co)ntinual learning framework.
arXiv Detail & Related papers (2024-01-05T22:22:13Z) - Less Can Be More: Unsupervised Graph Pruning for Large-scale Dynamic
Graphs [42.864057751606616]
We propose and study the problem of unsupervised graph pruning on dynamic graphs.
Our proposed STEP framework learns to remove potentially redundant edges from input dynamic graphs.
Our results on three real-world datasets demonstrate the advantages on improving the efficacy, robustness, and efficiency of GNNs.
arXiv Detail & Related papers (2023-05-18T03:23:53Z) - GIF: A General Graph Unlearning Strategy via Influence Function [63.52038638220563]
Graph Influence Function (GIF) is a model-agnostic unlearning method that can efficiently and accurately estimate parameter changes in response to a $epsilon$-mass perturbation in deleted data.
We conduct extensive experiments on four representative GNN models and three benchmark datasets to justify GIF's superiority in terms of unlearning efficacy, model utility, and unlearning efficiency.
arXiv Detail & Related papers (2023-04-06T03:02:54Z) - State of the Art and Potentialities of Graph-level Learning [54.68482109186052]
Graph-level learning has been applied to many tasks including comparison, regression, classification, and more.
Traditional approaches to learning a set of graphs rely on hand-crafted features, such as substructures.
Deep learning has helped graph-level learning adapt to the growing scale of graphs by extracting features automatically and encoding graphs into low-dimensional representations.
arXiv Detail & Related papers (2023-01-14T09:15:49Z) - ARIEL: Adversarial Graph Contrastive Learning [51.14695794459399]
ARIEL consistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks.
ARIEL is more robust in the face of adversarial attacks.
arXiv Detail & Related papers (2022-08-15T01:24:42Z) - Adversarial Graph Contrastive Learning with Information Regularization [51.14695794459399]
Contrastive learning is an effective method in graph representation learning.
Data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples.
We propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL)
It consistently outperforms the current graph contrastive learning methods in the node classification task over various real-world datasets.
arXiv Detail & Related papers (2022-02-14T05:54:48Z) - Dynamic Graph Learning-Neural Network for Multivariate Time Series
Modeling [2.3022070933226217]
We propose a novel framework, namely static- and dynamic-graph learning-neural network (GL)
The model acquires static and dynamic graph matrices from data to model long-term and short-term patterns respectively.
It achieves state-of-the-art performance on almost all datasets.
arXiv Detail & Related papers (2021-12-06T08:19:15Z) - Learning to Evolve on Dynamic Graphs [5.1521870302904125]
Learning to Evolve on Dynamic Graphs (LEDG) is a novel algorithm that jointly learns graph information and time information.
LEDG is model-agnostic and can train any message passing based graph neural network (GNN) on dynamic graphs.
arXiv Detail & Related papers (2021-11-13T04:09:30Z) - Online Graph Learning in Dynamic Environments [5.222057229549077]
This paper focuses on learning graphs in the case of sequential data in dynamic environments.
For sequential data, we develop an online version of classic batch graph learning method.
arXiv Detail & Related papers (2021-10-11T06:24:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.