Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering

• URL: http://arxiv.org/abs/2210.10462v2
• Date: Wed, 12 Apr 2023 12:20:55 GMT
• Title: Self-supervised Heterogeneous Graph Pre-training Based on Structural Clustering
• Authors: Yaming Yang, Ziyu Guan, Zhe Wang, Wei Zhao, Cai Xu, Weigang Lu, Jianbin Huang
• Abstract summary: We present SHGP, a novel Self-supervised Heterogeneous Graph Pre-training approach. It does not need to generate any positive examples or negative examples. It is superior to state-of-the-art unsupervised baselines and even semi-supervised baselines.
• Score: 20.985559149384795
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent self-supervised pre-training methods on Heterogeneous Information Networks (HINs) have shown promising competitiveness over traditional semi-supervised Heterogeneous Graph Neural Networks (HGNNs). Unfortunately, their performance heavily depends on careful customization of various strategies for generating high-quality positive examples and negative examples, which notably limits their flexibility and generalization ability. In this work, we present SHGP, a novel Self-supervised Heterogeneous Graph Pre-training approach, which does not need to generate any positive examples or negative examples. It consists of two modules that share the same attention-aggregation scheme. In each iteration, the Att-LPA module produces pseudo-labels through structural clustering, which serve as the self-supervision signals to guide the Att-HGNN module to learn object embeddings and attention coefficients. The two modules can effectively utilize and enhance each other, promoting the model to learn discriminative embeddings. Extensive experiments on four real-world datasets demonstrate the superior effectiveness of SHGP against state-of-the-art unsupervised baselines and even semi-supervised baselines. We release our source code at: https://github.com/kepsail/SHGP.

Related papers

• Language Models are Graph Learners [70.14063765424012]
  Language Models (LMs) are challenging the dominance of domain-specific models, including Graph Neural Networks (GNNs) and Graph Transformers (GTs) We propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the-art GNNs on node classification tasks.
  arXiv  Detail & Related papers  (2024-10-03T08:27:54Z)
• Self-Supervised Contrastive Graph Clustering Network via Structural Information Fusion [15.293684479404092]
  We propose a novel deep graph clustering method called CGCN. Our approach introduces contrastive signals and deep structural information into the pre-training process. Our method has been experimentally validated on multiple real-world graph datasets.
  arXiv  Detail & Related papers  (2024-08-08T09:49:26Z)
• Deep Contrastive Graph Learning with Clustering-Oriented Guidance [61.103996105756394]
  Graph Convolutional Network (GCN) has exhibited remarkable potential in improving graph-based clustering. Models estimate an initial graph beforehand to apply GCN. Deep Contrastive Graph Learning (DCGL) model is proposed for general data clustering.
  arXiv  Detail & Related papers  (2024-02-25T07:03:37Z)
• Empowering Dual-Level Graph Self-Supervised Pretraining with Motif Discovery [28.38130326794833]
  We introduce Dual-level Graph self-supervised Pretraining with Motif discovery (DGPM) DGPM orchestrates node-level and subgraph-level pretext tasks. Experiments on 15 datasets validate DGPM's effectiveness and generalizability.
  arXiv  Detail & Related papers  (2023-12-19T08:09:36Z)
• Homophily-enhanced Structure Learning for Graph Clustering [19.586401211161846]
  Graph structure learning allows refining the input graph by adding missing links and removing spurious connections. Previous endeavors in graph structure learning have predominantly centered around supervised settings. We propose a novel method called textbfhomophily-enhanced structure textbflearning for graph clustering (HoLe)
  arXiv  Detail & Related papers  (2023-08-10T02:53:30Z)
• Interpolation-based Correlation Reduction Network for Semi-Supervised Graph Learning [49.94816548023729]
  We propose a novel graph contrastive learning method, termed Interpolation-based Correlation Reduction Network (ICRN) In our method, we improve the discriminative capability of the latent feature by enlarging the margin of decision boundaries. By combining the two settings, we extract rich supervision information from both the abundant unlabeled nodes and the rare yet valuable labeled nodes for discnative representation learning.
  arXiv  Detail & Related papers  (2022-06-06T14:26:34Z)
• Deep Attention-guided Graph Clustering with Dual Self-supervision [49.040136530379094]
  We propose a novel method, namely deep attention-guided graph clustering with dual self-supervision (DAGC) We develop a dual self-supervision solution consisting of a soft self-supervision strategy with a triplet Kullback-Leibler divergence loss and a hard self-supervision strategy with a pseudo supervision loss. Our method consistently outperforms state-of-the-art methods on six benchmark datasets.
  arXiv  Detail & Related papers  (2021-11-10T06:53:03Z)
• Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach [80.8446673089281]
  We propose a new learning paradigm with graph representation and learning. Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
  arXiv  Detail & Related papers  (2021-10-09T09:02:45Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.