Graph Few-shot Class-incremental Learning
- URL: http://arxiv.org/abs/2112.12819v1
- Date: Thu, 23 Dec 2021 19:46:07 GMT
- Title: Graph Few-shot Class-incremental Learning
- Authors: Zhen Tan, Kaize Ding, Ruocheng Guo, Huan Liu
- Abstract summary: The ability to incrementally learn new classes is vital to all real-world artificial intelligence systems.
In this paper, we investigate the challenging yet practical problem, Graph Few-shot Class-incremental (Graph FCL) problem.
We put forward a Graph Pseudo Incremental Learning paradigm by sampling tasks recurrently from the base classes.
We present a task-sensitive regularizer calculated from task-level attention and node class prototypes to mitigate overfitting onto either novel or base classes.
- Score: 25.94168397283495
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The ability to incrementally learn new classes is vital to all real-world
artificial intelligence systems. A large portion of high-impact applications
like social media, recommendation systems, E-commerce platforms, etc. can be
represented by graph models. In this paper, we investigate the challenging yet
practical problem, Graph Few-shot Class-incremental (Graph FCL) problem, where
the graph model is tasked to classify both newly encountered classes and
previously learned classes. Towards that purpose, we put forward a Graph Pseudo
Incremental Learning paradigm by sampling tasks recurrently from the base
classes, so as to produce an arbitrary number of training episodes for our
model to practice the incremental learning skill. Furthermore, we design a
Hierarchical-Attention-based Graph Meta-learning framework, HAG-Meta. We
present a task-sensitive regularizer calculated from task-level attention and
node class prototypes to mitigate overfitting onto either novel or base
classes. To employ the topological knowledge, we add a node-level attention
module to adjust the prototype representation. Our model not only achieves
greater stability of old knowledge consolidation, but also acquires
advantageous adaptability to new knowledge with very limited data samples.
Extensive experiments on three real-world datasets, including Amazon-clothing,
Reddit, and DBLP, show that our framework demonstrates remarkable advantages in
comparison with the baseline and other related state-of-the-art methods.
Related papers
- OpenGraph: Towards Open Graph Foundation Models [20.401374302429627]
We develop a general graph foundation model to understand the complex topological patterns present in diverse graph data.
We propose a unified graph tokenizer to adapt our graph model to generalize well on unseen graph data.
We also develop a scalable graph transformer, which effectively captures node-wise dependencies within the global topological context.
arXiv Detail & Related papers (2024-03-02T08:05:03Z) - UniGraph: Learning a Cross-Domain Graph Foundation Model From Natural
Language [41.722898353772656]
We present our UniGraph framework, designed to train a graph foundation model capable of generalizing to unseen graphs and tasks across diverse domains.
We propose a cascaded architecture of Language Models (LMs) and Graph Neural Networks (GNNs) as backbone networks with a self-supervised training objective based on Masked Graph Modeling (MGM)
Our comprehensive experiments across various graph learning tasks and domains demonstrate the model's effectiveness in self-supervised representation learning on unseen graphs, few-shot in-context transfer, and zero-shot transfer.
arXiv Detail & Related papers (2024-02-21T09:06:31Z) - RAHNet: Retrieval Augmented Hybrid Network for Long-tailed Graph
Classification [10.806893809269074]
We propose a novel framework called Retrieval Augmented Hybrid Network (RAHNet) to jointly learn a robust feature extractor and an unbiased classifier.
In the feature extractor training stage, we develop a graph retrieval module to search for relevant graphs that directly enrich the intra-class diversity for the tail classes.
We also innovatively optimize a category-centered supervised contrastive loss to obtain discriminative representations.
arXiv Detail & Related papers (2023-08-04T14:06:44Z) - SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [131.04781590452308]
We present SimTeG, a frustratingly Simple approach for Textual Graph learning.
We first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task.
We then generate node embeddings using the last hidden states of finetuned LM.
arXiv Detail & Related papers (2023-08-03T07:00:04Z) - Towards Unsupervised Deep Graph Structure Learning [67.58720734177325]
We propose an unsupervised graph structure learning paradigm, where the learned graph topology is optimized by data itself without any external guidance.
Specifically, we generate a learning target from the original data as an "anchor graph", and use a contrastive loss to maximize the agreement between the anchor graph and the learned graph.
arXiv Detail & Related papers (2022-01-17T11:57:29Z) - Towards Open-World Feature Extrapolation: An Inductive Graph Learning
Approach [80.8446673089281]
We propose a new learning paradigm with graph representation and learning.
Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
arXiv Detail & Related papers (2021-10-09T09:02:45Z) - Weakly-supervised Graph Meta-learning for Few-shot Node Classification [53.36828125138149]
We propose a new graph meta-learning framework -- Graph Hallucination Networks (Meta-GHN)
Based on a new robustness-enhanced episodic training, Meta-GHN is meta-learned to hallucinate clean node representations from weakly-labeled data.
Extensive experiments demonstrate the superiority of Meta-GHN over existing graph meta-learning studies.
arXiv Detail & Related papers (2021-06-12T22:22:10Z) - Few-Shot Incremental Learning with Continually Evolved Classifiers [46.278573301326276]
Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points.
The difficulty lies in that limited data from new classes not only lead to significant overfitting issues but also exacerbate the notorious catastrophic forgetting problems.
We propose a Continually Evolved CIF ( CEC) that employs a graph model to propagate context information between classifiers for adaptation.
arXiv Detail & Related papers (2021-04-07T10:54:51Z) - Model-Agnostic Graph Regularization for Few-Shot Learning [60.64531995451357]
We present a comprehensive study on graph embedded few-shot learning.
We introduce a graph regularization approach that allows a deeper understanding of the impact of incorporating graph information between labels.
Our approach improves the performance of strong base learners by up to 2% on Mini-ImageNet and 6.7% on ImageNet-FS.
arXiv Detail & Related papers (2021-02-14T05:28:13Z) - Tensor Graph Convolutional Networks for Multi-relational and Robust
Learning [74.05478502080658]
This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor.
The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
arXiv Detail & Related papers (2020-03-15T02:33:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.