Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach

• URL: http://arxiv.org/abs/2410.10341v2
• Date: Mon, 28 Oct 2024 00:01:22 GMT
• Title: Replay-and-Forget-Free Graph Class-Incremental Learning: A Task Profiling and Prompting Approach
• Authors: Chaoxi Niu, Guansong Pang, Ling Chen, Bing Liu,
• Abstract summary: Class-incremental learning (CIL) aims to continually learn a sequence of tasks, with each task consisting of a set of unique classes. The key characteristic of CIL lies in the absence of task identifiers (IDs) during inference. We show theoretically that accurate task ID prediction on graph data can be achieved by a Laplacian smoothing-based graph task profiling approach.
• Score: 28.194940062243003
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Class-incremental learning (CIL) aims to continually learn a sequence of tasks, with each task consisting of a set of unique classes. Graph CIL (GCIL) follows the same setting but needs to deal with graph tasks (e.g., node classification in a graph). The key characteristic of CIL lies in the absence of task identifiers (IDs) during inference, which causes a significant challenge in separating classes from different tasks (i.e., inter-task class separation). Being able to accurately predict the task IDs can help address this issue, but it is a challenging problem. In this paper, we show theoretically that accurate task ID prediction on graph data can be achieved by a Laplacian smoothing-based graph task profiling approach, in which each graph task is modeled by a task prototype based on Laplacian smoothing over the graph. It guarantees that the task prototypes of the same graph task are nearly the same with a large smoothing step, while those of different tasks are distinct due to differences in graph structure and node attributes. Further, to avoid the catastrophic forgetting of the knowledge learned in previous graph tasks, we propose a novel graph prompting approach for GCIL which learns a small discriminative graph prompt for each task, essentially resulting in a separate classification model for each task. The prompt learning requires the training of a single graph neural network (GNN) only once on the first task, and no data replay is required thereafter, thereby obtaining a GCIL model being both replay-free and forget-free. Extensive experiments on four GCIL benchmarks show that i) our task prototype-based method can achieve 100% task ID prediction accuracy on all four datasets, ii) our GCIL model significantly outperforms state-of-the-art competing methods by at least 18% in average CIL accuracy, and iii) our model is fully free of forgetting on the four datasets.

Related papers

• Exploring Task Unification in Graph Representation Learning via Generative Approach [12.983429541410617]
  Graphs are ubiquitous in real-world scenarios and encompass a diverse range of tasks, from node-, edge-, and graph-level tasks to transfer learning. Recent endeavors aim to design a unified framework capable of generalizing across multiple graph tasks. Among these, graph autoencoders (GAEs) have demonstrated their potential in effectively addressing various graph tasks. We propose GA2E, a unified adversarially masked autoencoder capable of addressing the above challenges seamlessly.
  arXiv  Detail & Related papers  (2024-03-21T12:14:02Z)
• All in One: Multi-Task Prompting for Graph Neural Networks (Extended Abstract) [30.457491401821652]
  This paper is an extended abstract of our original work published in KDD23, where we won the best research paper award. It introduces a novel approach to bridging the gap between pre-trained graph models and the diverse tasks they're applied to.
  arXiv  Detail & Related papers  (2024-03-11T16:04:58Z)
• One for All: Towards Training One Graph Model for All Classification Tasks [61.656962278497225]
  A unified model for various graph tasks remains underexplored, primarily due to the challenges unique to the graph learning domain. We propose textbfOne for All (OFA), the first general framework that can use a single graph model to address the above challenges. OFA performs well across different tasks, making it the first general-purpose across-domains classification model on graphs.
  arXiv  Detail & Related papers  (2023-09-29T21:15:26Z)
• SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [131.04781590452308]
  We present SimTeG, a frustratingly Simple approach for Textual Graph learning. We first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task. We then generate node embeddings using the last hidden states of finetuned LM.
  arXiv  Detail & Related papers  (2023-08-03T07:00:04Z)
• All in One: Multi-task Prompting for Graph Neural Networks [30.457491401821652]
  We propose a novel multi-task prompting method for graph models. We first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. We then study the task space of various graph applications and reformulate downstream problems to the graph-level task.
  arXiv  Detail & Related papers  (2023-07-04T06:27:31Z)
• Unsupervised Task Graph Generation from Instructional Video Transcripts [53.54435048879365]
  We consider a setting where text transcripts of instructional videos performing a real-world activity are provided. The goal is to identify the key steps relevant to the task as well as the dependency relationship between these key steps. We propose a novel task graph generation approach that combines the reasoning capabilities of instruction-tuned language models along with clustering and ranking components.
  arXiv  Detail & Related papers  (2023-02-17T22:50:08Z)
• Self-supervised Learning for Heterogeneous Graph via Structure Information based on Metapath [9.757299837675204]
  Self-supervised representation learning is a potential approach to tackle this problem. In this paper, we propose a SElfsupervised learning method for heterogeneous graph via Structure Information based on Metapath. In order to predict jump number, SESIM uses data itself to generate labels, avoiding time-consuming manual labeling.
  arXiv  Detail & Related papers  (2022-09-09T10:06:18Z)
• DOTIN: Dropping Task-Irrelevant Nodes for GNNs [119.17997089267124]
  Recent graph learning approaches have introduced the pooling strategy to reduce the size of graphs for learning. We design a new approach called DOTIN (underlineDrunderlineopping underlineTask-underlineIrrelevant underlineNodes) to reduce the size of graphs. Our method speeds up GAT by about 50% on graph-level tasks including graph classification and graph edit distance.
  arXiv  Detail & Related papers  (2022-04-28T12:00:39Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training [62.73470368851127]
  Graph representation learning has emerged as a powerful technique for addressing real-world problems. We design Graph Contrastive Coding -- a self-supervised graph neural network pre-training framework. We conduct experiments on three graph learning tasks and ten graph datasets.
  arXiv  Detail & Related papers  (2020-06-17T16:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.