Hierarchical Contrastive Learning Enhanced Heterogeneous Graph Neural Network

• URL: http://arxiv.org/abs/2304.12228v2
• Date: Tue, 5 Mar 2024 07:11:14 GMT
• Title: Hierarchical Contrastive Learning Enhanced Heterogeneous Graph Neural Network
• Authors: Nian Liu, Xiao Wang, Hui Han, Chuan Shi
• Abstract summary: Heterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN) Recently, contrastive learning, a self-supervised method, becomes one of the most exciting learning paradigms and shows great potential when there are no labels. In this paper, we study the problem of self-supervised HGNNs and propose a novel co-contrastive learning mechanism for HGNNs, named HeCo.
• Score: 59.860534520941485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Heterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN). However, most HGNNs follow a semi-supervised learning manner, which notably limits their wide use in reality since labels are usually scarce in real applications. Recently, contrastive learning, a self-supervised method, becomes one of the most exciting learning paradigms and shows great potential when there are no labels. In this paper, we study the problem of self-supervised HGNNs and propose a novel co-contrastive learning mechanism for HGNNs, named HeCo. Different from traditional contrastive learning which only focuses on contrasting positive and negative samples, HeCo employs cross-view contrastive mechanism. Specifically, two views of a HIN (network schema and meta-path views) are proposed to learn node embeddings, so as to capture both of local and high-order structures simultaneously. Then the cross-view contrastive learning, as well as a view mask mechanism, is proposed, which is able to extract the positive and negative embeddings from two views. This enables the two views to collaboratively supervise each other and finally learn high-level node embeddings. Moreover, to further boost the performance of HeCo, two additional methods are designed to generate harder negative samples with high quality. Besides the invariant factors, view-specific factors complementally provide the diverse structure information between different nodes, which also should be contained into the final embeddings. Therefore, we need to further explore each view independently and propose a modified model, called HeCo++. Specifically, HeCo++ conducts hierarchical contrastive learning, including cross-view and intra-view contrasts, which aims to enhance the mining of respective structures.

Related papers

• Generative-Enhanced Heterogeneous Graph Contrastive Learning [11.118517297006894]
  Heterogeneous Graphs (HGs) can effectively model complex relationships in the real world by multi-type nodes and edges. In recent years, inspired by self-supervised learning, contrastive Heterogeneous Graphs Neural Networks (HGNNs) have shown great potential by utilizing data augmentation and contrastive discriminators for downstream tasks. We propose a novel Generative-Enhanced Heterogeneous Graph Contrastive Learning (GHGCL)
  arXiv  Detail & Related papers  (2024-04-03T15:31:18Z)
• Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias [75.44877675117749]
  We propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs. Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph datasets Benchmark.
  arXiv  Detail & Related papers  (2023-09-26T13:09:43Z)
• Self-Contrastive Graph Diffusion Network [1.14219428942199]
  We propose a novel framework called the Self-Contrastive Graph Diffusion Network (SCGDN) Our framework consists of two main components: the Attentional Module (AttM) and the Diffusion Module (DiFM) Unlike existing methodologies, SCGDN is an augmentation-free approach that avoids "sampling bias" and semantic drift.
  arXiv  Detail & Related papers  (2023-07-27T04:00:23Z)
• STERLING: Synergistic Representation Learning on Bipartite Graphs [78.86064828220613]
  A fundamental challenge of bipartite graph representation learning is how to extract node embeddings. Most recent bipartite graph SSL methods are based on contrastive learning which learns embeddings by discriminating positive and negative node pairs. We introduce a novel synergistic representation learning model (STERLING) to learn node embeddings without negative node pairs.
  arXiv  Detail & Related papers  (2023-01-25T03:21:42Z)
• Interpolation-based Correlation Reduction Network for Semi-Supervised Graph Learning [49.94816548023729]
  We propose a novel graph contrastive learning method, termed Interpolation-based Correlation Reduction Network (ICRN) In our method, we improve the discriminative capability of the latent feature by enlarging the margin of decision boundaries. By combining the two settings, we extract rich supervision information from both the abundant unlabeled nodes and the rare yet valuable labeled nodes for discnative representation learning.
  arXiv  Detail & Related papers  (2022-06-06T14:26:34Z)
• Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning [102.9138736545956]
  Heterogeneous graph neural network (HGNN) is a very popular technique for the modeling and analysis of heterogeneous graphs. We develop for the first time a novel and robust heterogeneous graph contrastive learning approach, namely HGCL, which introduces two views on respective guidance of node attributes and graph topologies. In this new approach, we adopt distinct but most suitable attribute and topology fusion mechanisms in the two views, which are conducive to mining relevant information in attributes and topologies separately.
  arXiv  Detail & Related papers  (2022-04-30T12:57:02Z)
• Deep Contrastive Learning for Multi-View Network Embedding [20.035449838566503]
  Multi-view network embedding aims at projecting nodes in the network to low-dimensional vectors. Most contrastive learning-based methods mostly rely on high-quality graph embedding. We design a novel node-to-node Contrastive learning framework for Multi-view network Embedding (CREME)
  arXiv  Detail & Related papers  (2021-08-16T06:29:18Z)
• Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learning [38.062495223111355]
  Heterogeneous graph neural networks (HGNNs) as an emerging technique have shown superior capacity of dealing with heterogeneous information network (HIN) In this paper, we study the problem of self-supervised HGNNs and propose a novel co-contrastive learning mechanism for HGNNs, named HeCo.
  arXiv  Detail & Related papers  (2021-05-19T13:15:03Z)
• Self-supervised Graph Learning for Recommendation [69.98671289138694]
  We explore self-supervised learning on user-item graph for recommendation. An auxiliary self-supervised task reinforces node representation learning via self-discrimination. Empirical studies on three benchmark datasets demonstrate the effectiveness of SGL.
  arXiv  Detail & Related papers  (2020-10-21T06:35:26Z)
• GraphCL: Contrastive Self-Supervised Learning of Graph Representations [20.439666392958284]
  We propose Graph Contrastive Learning (GraphCL), a general framework for learning node representations in a self supervised manner. We use graph neural networks to produce two representations of the same node and leverage a contrastive learning loss to maximize agreement between them. In both transductive and inductive learning setups, we demonstrate that our approach significantly outperforms the state-of-the-art in unsupervised learning on a number of node classification benchmarks.
  arXiv  Detail & Related papers  (2020-07-15T22:36:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.