GraphPrompter: Multi-stage Adaptive Prompt Optimization for Graph In-Context Learning

• URL: http://arxiv.org/abs/2505.02027v1
• Date: Sun, 04 May 2025 08:30:00 GMT
• Title: GraphPrompter: Multi-stage Adaptive Prompt Optimization for Graph In-Context Learning
• Authors: Rui Lv, Zaixi Zhang, Kai Zhang, Qi Liu, Weibo Gao, Jiawei Liu, Jiaxia Yan, Linan Yue, Fangzhou Yao,
• Abstract summary: The key to graph in-context learning is to perform downstream graphs conditioned on chosen prompt examples.<n>Existing methods randomly select subgraphs or edges as prompts, leading to noisy graph prompts and inferior model performance.<n>We develop a multi-stage adaptive prompt optimization method GraphPrompter.<n>Our approach surpasses the state-of-the-art baselines by over 8%.
• Score: 18.254759409121956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph In-Context Learning, with the ability to adapt pre-trained graph models to novel and diverse downstream graphs without updating any parameters, has gained much attention in the community. The key to graph in-context learning is to perform downstream graphs conditioned on chosen prompt examples. Existing methods randomly select subgraphs or edges as prompts, leading to noisy graph prompts and inferior model performance. Additionally, due to the gap between pre-training and testing graphs, when the number of classes in the testing graphs is much greater than that in the training, the in-context learning ability will also significantly deteriorate. To tackle the aforementioned challenges, we develop a multi-stage adaptive prompt optimization method GraphPrompter, which optimizes the entire process of generating, selecting, and using graph prompts for better in-context learning capabilities. Firstly, Prompt Generator introduces a reconstruction layer to highlight the most informative edges and reduce irrelevant noise for graph prompt construction. Furthermore, in the selection stage, Prompt Selector employs the $k$-nearest neighbors algorithm and pre-trained selection layers to dynamically choose appropriate samples and minimize the influence of irrelevant prompts. Finally, we leverage a Prompt Augmenter with a cache replacement strategy to enhance the generalization capability of the pre-trained model on new datasets. Extensive experiments show that GraphPrompter effectively enhances the in-context learning ability of graph models. On average across all the settings, our approach surpasses the state-of-the-art baselines by over 8%. Our code is released at https://github.com/karin0018/GraphPrompter.

Related papers

• Graph Prompting for Graph Learning Models: Recent Advances and Future Directions [75.7773954442738]
  "Pre-training, adaptation" scheme first pre-trains graph learning models on unlabeled graph data in a self-supervised manner.<n> graph prompting emerges as a promising approach that learns trainable prompts while keeping the pre-trained graph learning models unchanged.
  arXiv  Detail & Related papers  (2025-06-10T01:27:19Z)
• Edge Prompt Tuning for Graph Neural Networks [40.62424370491229]
  We propose EdgePrompt, a simple yet effective graph prompt tuning method from the perspective of edges.<n>Our method is compatible with prevalent GNN architectures pre-trained under various pre-training strategies.
  arXiv  Detail & Related papers  (2025-03-02T06:07:54Z)
• Instance-Aware Graph Prompt Learning [71.26108600288308]
  We introduce Instance-Aware Graph Prompt Learning (IA-GPL) in this paper. The process involves generating intermediate prompts for each instance using a lightweight architecture. Experiments conducted on multiple datasets and settings showcase the superior performance of IA-GPL compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-11-26T18:38:38Z)
• ProG: A Graph Prompt Learning Benchmark [17.229372585695092]
  Graph prompt learning emerges as a promising alternative to 'Pre-train & Fine-tune' We introduce the first comprehensive benchmark for graph prompt learning. We present 'ProG', an easy-to-use open-source library that streamlines the execution of various graph prompt models.
  arXiv  Detail & Related papers  (2024-06-08T04:17:48Z)
• Self-Pro: A Self-Prompt and Tuning Framework for Graph Neural Networks [10.794305560114903]
  Self-Prompt is a prompting framework for graphs based on the model and data itself. We introduce asymmetric graph contrastive learning for pretext to address heterophily and align the objectives of pretext and downstream tasks. We conduct extensive experiments on 11 benchmark datasets to demonstrate its superiority.
  arXiv  Detail & Related papers  (2023-10-16T12:58:04Z)
• Deep Prompt Tuning for Graph Transformers [55.2480439325792]
  Fine-tuning is resource-intensive and requires storing multiple copies of large models. We propose a novel approach called deep graph prompt tuning as an alternative to fine-tuning. By freezing the pre-trained parameters and only updating the added tokens, our approach reduces the number of free parameters and eliminates the need for multiple model copies.
  arXiv  Detail & Related papers  (2023-09-18T20:12:17Z)
• All in One: Multi-task Prompting for Graph Neural Networks [30.457491401821652]
  We propose a novel multi-task prompting method for graph models. We first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. We then study the task space of various graph applications and reformulate downstream problems to the graph-level task.
  arXiv  Detail & Related papers  (2023-07-04T06:27:31Z)
• PRODIGY: Enabling In-context Learning Over Graphs [112.19056551153454]
  In-context learning is the ability of a pretrained model to adapt to novel and diverse downstream tasks. We develop PRODIGY, the first pretraining framework that enables in-context learning over graphs.
  arXiv  Detail & Related papers  (2023-05-21T23:16:30Z)
• State of the Art and Potentialities of Graph-level Learning [54.68482109186052]
  Graph-level learning has been applied to many tasks including comparison, regression, classification, and more. Traditional approaches to learning a set of graphs rely on hand-crafted features, such as substructures. Deep learning has helped graph-level learning adapt to the growing scale of graphs by extracting features automatically and encoding graphs into low-dimensional representations.
  arXiv  Detail & Related papers  (2023-01-14T09:15:49Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• Adversarial Graph Contrastive Learning with Information Regularization [51.14695794459399]
  Contrastive learning is an effective method in graph representation learning. Data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples. We propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL) It consistently outperforms the current graph contrastive learning methods in the node classification task over various real-world datasets.
  arXiv  Detail & Related papers  (2022-02-14T05:54:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.