GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks

• URL: http://arxiv.org/abs/2302.08043v1
• Date: Thu, 16 Feb 2023 02:51:38 GMT
• Title: GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks
• Authors: Zemin Liu, Xingtong Yu, Yuan Fang, Xinming Zhang
• Abstract summary: GraphPrompt is a novel pre-training and prompting framework on graphs. It unifies pre-training and downstream tasks into a common task template. It also employs a learnable prompt to assist a downstream task in locating the most relevant knowledge from the pre-train model.
• Score: 16.455234748896157
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graphs can model complex relationships between objects, enabling a myriad of Web applications such as online page/article classification and social recommendation. While graph neural networks(GNNs) have emerged as a powerful tool for graph representation learning, in an end-to-end supervised setting, their performance heavily rely on a large amount of task-specific supervision. To reduce labeling requirement, the "pre-train, fine-tune" and "pre-train, prompt" paradigms have become increasingly common. In particular, prompting is a popular alternative to fine-tuning in natural language processing, which is designed to narrow the gap between pre-training and downstream objectives in a task-specific manner. However, existing study of prompting on graphs is still limited, lacking a universal treatment to appeal to different downstream tasks. In this paper, we propose GraphPrompt, a novel pre-training and prompting framework on graphs. GraphPrompt not only unifies pre-training and downstream tasks into a common task template, but also employs a learnable prompt to assist a downstream task in locating the most relevant knowledge from the pre-train model in a task-specific manner. Finally, we conduct extensive experiments on five public datasets to evaluate and analyze GraphPrompt.

Related papers

• Can Graph Learning Improve Planning in LLM-based Agents? [61.47027387839096]
  Task planning in language agents is emerging as an important research topic alongside the development of large language models (LLMs) In this paper, we explore graph learning-based methods for task planning, a direction that is to the prevalent focus on prompt design. Our interest in graph learning stems from a theoretical discovery: the biases of attention and auto-regressive loss impede LLMs' ability to effectively navigate decision-making on graphs.
  arXiv  Detail & Related papers  (2024-05-29T14:26:24Z)
• MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs [33.2696184519275]
  MultiGPrompt is a novel multi-task pre-training and prompting framework for graph representation learning. We propose a dual-prompt mechanism consisting of composed and open prompts to leverage task-specific and global pre-training knowledge.
  arXiv  Detail & Related papers  (2023-11-28T02:36:53Z)
• Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs [20.406549548630156]
  GraphPrompt is a novel pre-training and prompting framework on graphs. It unifies pre-training and downstream tasks into a common task template. It also employs a learnable prompt to assist a downstream task in locating the most relevant knowledge from the pre-trained model.
  arXiv  Detail & Related papers  (2023-11-26T14:35:28Z)
• HetGPT: Harnessing the Power of Prompt Tuning in Pre-Trained Heterogeneous Graph Neural Networks [24.435068514392487]
  HetGPT is a post-training prompting framework for graph neural networks. It improves the performance of state-of-the-art HGNNs on semi-supervised node classification.
  arXiv  Detail & Related papers  (2023-10-23T19:35:57Z)
• ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt [67.8934749027315]
  We propose a unified framework for graph hybrid pre-training which injects the task identification and position identification into GNNs. We also propose a novel pre-training paradigm based on a group of $k$-nearest neighbors.
  arXiv  Detail & Related papers  (2023-10-23T12:11:13Z)
• Deep Prompt Tuning for Graph Transformers [55.2480439325792]
  Fine-tuning is resource-intensive and requires storing multiple copies of large models. We propose a novel approach called deep graph prompt tuning as an alternative to fine-tuning. By freezing the pre-trained parameters and only updating the added tokens, our approach reduces the number of free parameters and eliminates the need for multiple model copies.
  arXiv  Detail & Related papers  (2023-09-18T20:12:17Z)
• SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [131.04781590452308]
  We present SimTeG, a frustratingly Simple approach for Textual Graph learning. We first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task. We then generate node embeddings using the last hidden states of finetuned LM.
  arXiv  Detail & Related papers  (2023-08-03T07:00:04Z)
• All in One: Multi-task Prompting for Graph Neural Networks [30.457491401821652]
  We propose a novel multi-task prompting method for graph models. We first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. We then study the task space of various graph applications and reformulate downstream problems to the graph-level task.
  arXiv  Detail & Related papers  (2023-07-04T06:27:31Z)
• A Survey of Graph Prompting Methods: Techniques, Applications, and Challenges [25.32529044997131]
  "Pre-train, prompt, predict training" has gained popularity as a way to learn generalizable models with limited labeled data. The design of prompts could be a challenging and time-consuming process in complex tasks. This survey will bridge the gap between graphs and prompt design to facilitate future methodology development.
  arXiv  Detail & Related papers  (2023-03-13T16:49:43Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training [62.73470368851127]
  Graph representation learning has emerged as a powerful technique for addressing real-world problems. We design Graph Contrastive Coding -- a self-supervised graph neural network pre-training framework. We conduct experiments on three graph learning tasks and ten graph datasets.
  arXiv  Detail & Related papers  (2020-06-17T16:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.