Refining Latent Representations: A Generative SSL Approach for Heterogeneous Graph Learning

• URL: http://arxiv.org/abs/2310.11102v4
• Date: Sat, 20 Apr 2024 07:34:42 GMT
• Title: Refining Latent Representations: A Generative SSL Approach for Heterogeneous Graph Learning
• Authors: Yulan Hu, Zhirui Yang, Sheng Ouyang, Yong Liu,
• Abstract summary: We propose HGVAE, a generative SSL method specially designed for Heterogeneous Graph Learning (HGL) Instead of focusing on designing complex strategies to capture heterogeneity, HGVAE centers on refining the latent representation. As a pioneer in applying generative SSL for HGL, HGVAE refines the latent representation, thereby compelling the model to learn high-quality representations.
• Score: 9.686218058331061
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-Supervised Learning (SSL) has shown significant potential and has garnered increasing interest in graph learning. However, particularly for generative SSL methods, its potential in Heterogeneous Graph Learning (HGL) remains relatively underexplored. Generative SSL utilizes an encoder to map the input graph into a latent representation and a decoder to recover the input graph from the latent representation. Previous HGL SSL methods generally design complex strategies to capture graph heterogeneity, which heavily rely on contrastive view construction strategies that are often non-trivial. Yet, refining the latent representation in generative SSL can effectively improve graph learning results. In this study, we propose HGVAE, a generative SSL method specially designed for HGL. Instead of focusing on designing complex strategies to capture heterogeneity, HGVAE centers on refining the latent representation. Specifically, HGVAE innovatively develops a contrastive task based on the latent representation. To ensure the hardness of negative samples, we develop a progressive negative sample generation (PNSG) mechanism that leverages the ability of Variational Inference (VI) to generate high-quality negative samples. As a pioneer in applying generative SSL for HGL, HGVAE refines the latent representation, thereby compelling the model to learn high-quality representations. Compared with various state-of-the-art (SOTA) baselines, HGVAE achieves impressive results, thus validating its superiority.

Related papers

• Variational Graph Contrastive Learning [1.9950682531209158]
  Graph representation learning (GRL) is a fundamental task in machine learning, aiming to encode high-dimensional graph-structured data into low-dimensional vectors. In this work, we propose a novel Subgraph Gaussian Embedding Contrast (SGEC) method.
  arXiv  Detail & Related papers  (2024-11-11T17:23:07Z)
• Enhancing Graph Self-Supervised Learning with Graph Interplay [8.775644935074407]
  Graph Interplay (GIP) is an innovative and versatile approach that significantly enhances the performance equipped with various existing GSSL methods. GIP advocates introducing direct graph-level communications by random inter-graph edges within standard batches. Our empirical study demonstrates that GIP surpasses the performance of prevailing GSSL methods by significant margins.
  arXiv  Detail & Related papers  (2024-10-05T07:05:21Z)
• Disentangled Generative Graph Representation Learning [51.59824683232925]
  This paper introduces DiGGR (Disentangled Generative Graph Representation Learning), a self-supervised learning framework. It aims to learn latent disentangled factors and utilize them to guide graph mask modeling. Experiments on 11 public datasets for two different graph learning tasks demonstrate that DiGGR consistently outperforms many previous self-supervised methods.
  arXiv  Detail & Related papers  (2024-08-24T05:13:02Z)
• ExGRG: Explicitly-Generated Relation Graph for Self-Supervised Representation Learning [4.105236597768038]
  Self-supervised learning has emerged as a powerful technique in pre-training deep learning models. This paper introduces a novel non-contrastive SSL approach to Explicitly Generate a compositional Relation Graph.
  arXiv  Detail & Related papers  (2024-02-09T19:16:04Z)
• VIGraph: Generative Self-supervised Learning for Class-Imbalanced Node Classification [9.686218058331061]
  Class imbalance in graph data presents significant challenges for node classification. Existing methods, such as SMOTE-based approaches, exhibit limitations in constructing imbalanced graphs. We introduce VIGraph, a simple yet effective generative SSL approach that relies on the Variational GAE as the fundamental model.
  arXiv  Detail & Related papers  (2023-11-02T12:36:19Z)
• GraphMAE: Self-Supervised Masked Graph Autoencoders [52.06140191214428]
  We present a masked graph autoencoder GraphMAE that mitigates issues for generative self-supervised graph learning. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE--a simple graph autoencoder with our careful designs--can consistently generate outperformance over both contrastive and generative state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-05-22T11:57:08Z)
• Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning [102.9138736545956]
  Heterogeneous graph neural network (HGNN) is a very popular technique for the modeling and analysis of heterogeneous graphs. We develop for the first time a novel and robust heterogeneous graph contrastive learning approach, namely HGCL, which introduces two views on respective guidance of node attributes and graph topologies. In this new approach, we adopt distinct but most suitable attribute and topology fusion mechanisms in the two views, which are conducive to mining relevant information in attributes and topologies separately.
  arXiv  Detail & Related papers  (2022-04-30T12:57:02Z)
• Graph Structure Learning with Variational Information Bottleneck [70.62851953251253]
  We propose a novel Variational Information Bottleneck guided Graph Structure Learning framework, namely VIB-GSL. VIB-GSL learns an informative and compressive graph structure to distill the actionable information for specific downstream tasks.
  arXiv  Detail & Related papers  (2021-12-16T14:22:13Z)
• Parameterized Hypercomplex Graph Neural Networks for Graph Classification [1.1852406625172216]
  We develop graph neural networks that leverage the properties of hypercomplex feature transformation. In particular, in our proposed class of models, the multiplication rule specifying the algebra itself is inferred from the data during training. We test our proposed hypercomplex GNN on several open graph benchmark datasets and show that our models reach state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-30T18:01:06Z)
• Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning [64.98816284854067]
  Graph-based Semi-Supervised Learning (SSL) aims to transfer the labels of a handful of labeled data to the remaining massive unlabeled data via a graph. A novel GCN-based SSL algorithm is presented in this paper to enrich the supervision signals by utilizing both data similarities and graph structure.
  arXiv  Detail & Related papers  (2020-09-15T13:59:28Z)
• High-Fidelity Synthesis with Disentangled Representation [60.19657080953252]
  We propose an Information-Distillation Generative Adrial Network (ID-GAN) for disentanglement learning and high-fidelity synthesis. Our method learns disentangled representation using VAE-based models, and distills the learned representation with an additional nuisance variable to the separate GAN-based generator for high-fidelity synthesis. Despite the simplicity, we show that the proposed method is highly effective, achieving comparable image generation quality to the state-of-the-art methods using the disentangled representation.
  arXiv  Detail & Related papers  (2020-01-13T14:39:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.