Exploring Task Unification in Graph Representation Learning via Generative Approach

• URL: http://arxiv.org/abs/2403.14340v1
• Date: Thu, 21 Mar 2024 12:14:02 GMT
• Title: Exploring Task Unification in Graph Representation Learning via Generative Approach
• Authors: Yulan Hu, Sheng Ouyang, Zhirui Yang, Ge Chen, Junchen Wan, Xiao Wang, Yong Liu,
• Abstract summary: Graphs are ubiquitous in real-world scenarios and encompass a diverse range of tasks, from node-, edge-, and graph-level tasks to transfer learning. Recent endeavors aim to design a unified framework capable of generalizing across multiple graph tasks. Among these, graph autoencoders (GAEs) have demonstrated their potential in effectively addressing various graph tasks. We propose GA2E, a unified adversarially masked autoencoder capable of addressing the above challenges seamlessly.
• Score: 12.983429541410617
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Graphs are ubiquitous in real-world scenarios and encompass a diverse range of tasks, from node-, edge-, and graph-level tasks to transfer learning. However, designing specific tasks for each type of graph data is often costly and lacks generalizability. Recent endeavors under the "Pre-training + Fine-tuning" or "Pre-training + Prompt" paradigms aim to design a unified framework capable of generalizing across multiple graph tasks. Among these, graph autoencoders (GAEs), generative self-supervised models, have demonstrated their potential in effectively addressing various graph tasks. Nevertheless, these methods typically employ multi-stage training and require adaptive designs, which on one hand make it difficult to be seamlessly applied to diverse graph tasks and on the other hand overlook the negative impact caused by discrepancies in task objectives between the different stages. To address these challenges, we propose GA^2E, a unified adversarially masked autoencoder capable of addressing the above challenges seamlessly. Specifically, GA^2E proposes to use the subgraph as the meta-structure, which remains consistent across all graph tasks (ranging from node-, edge-, and graph-level to transfer learning) and all stages (both during training and inference). Further, GA^2E operates in a \textbf{"Generate then Discriminate"} manner. It leverages the masked GAE to reconstruct the input subgraph whilst treating it as a generator to compel the reconstructed graphs resemble the input subgraph. Furthermore, GA^2E introduces an auxiliary discriminator to discern the authenticity between the reconstructed (generated) subgraph and the input subgraph, thus ensuring the robustness of the graph representation through adversarial training mechanisms. We validate GA^2E's capabilities through extensive experiments on 21 datasets across four types of graph tasks.

Related papers

• Instance-Aware Graph Prompt Learning [71.26108600288308]
  We introduce Instance-Aware Graph Prompt Learning (IA-GPL) in this paper. The process involves generating intermediate prompts for each instance using a lightweight architecture. Experiments conducted on multiple datasets and settings showcase the superior performance of IA-GPL compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-11-26T18:38:38Z)
• MuseGraph: Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining [41.19687587548107]
  Graph Neural Networks (GNNs) need to be re-trained every time when applied to different graph tasks and datasets. We propose a novel framework MuseGraph, which seamlessly integrates the strengths of GNNs and Large Language Models (LLMs) Our experimental results demonstrate significant improvements in different graph tasks.
  arXiv  Detail & Related papers  (2024-03-02T09:27:32Z)
• Unifying Generation and Prediction on Graphs with Latent Graph Diffusion [24.505897569096476]
  We propose the first framework that enables solving graph learning tasks of all levels. We first formulate prediction tasks including regression and classification into a generic (conditional) generation framework. We then propose Latent Graph Diffusion (LGD), a generative model that can generate node, edge, and graph-level features of all categories simultaneously.
  arXiv  Detail & Related papers  (2024-02-04T15:03:47Z)
• One for All: Towards Training One Graph Model for All Classification Tasks [61.656962278497225]
  A unified model for various graph tasks remains underexplored, primarily due to the challenges unique to the graph learning domain. We propose textbfOne for All (OFA), the first general framework that can use a single graph model to address the above challenges. OFA performs well across different tasks, making it the first general-purpose across-domains classification model on graphs.
  arXiv  Detail & Related papers  (2023-09-29T21:15:26Z)
• SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [131.04781590452308]
  We present SimTeG, a frustratingly Simple approach for Textual Graph learning. We first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task. We then generate node embeddings using the last hidden states of finetuned LM.
  arXiv  Detail & Related papers  (2023-08-03T07:00:04Z)
• Multi-View Graph Representation Learning Beyond Homophily [2.601278669926709]
  Unsupervised graph representation learning(GRL) aims to distill diverse graph information into task-agnostic embeddings without label supervision. A novel framework, denoted as Multi-view Graph(MVGE) is proposed, and a set of key designs are identified.
  arXiv  Detail & Related papers  (2023-04-15T08:35:49Z)
• GraphMAE: Self-Supervised Masked Graph Autoencoders [52.06140191214428]
  We present a masked graph autoencoder GraphMAE that mitigates issues for generative self-supervised graph learning. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE--a simple graph autoencoder with our careful designs--can consistently generate outperformance over both contrastive and generative state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-05-22T11:57:08Z)
• A Robust and Generalized Framework for Adversarial Graph Embedding [73.37228022428663]
  We propose a robust framework for adversarial graph embedding, named AGE. AGE generates the fake neighbor nodes as the enhanced negative samples from the implicit distribution. Based on this framework, we propose three models to handle three types of graph data.
  arXiv  Detail & Related papers  (2021-05-22T07:05:48Z)
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training [62.73470368851127]
  Graph representation learning has emerged as a powerful technique for addressing real-world problems. We design Graph Contrastive Coding -- a self-supervised graph neural network pre-training framework. We conduct experiments on three graph learning tasks and ten graph datasets.
  arXiv  Detail & Related papers  (2020-06-17T16:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.