Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning

• URL: http://arxiv.org/abs/2108.13886v1
• Date: Tue, 31 Aug 2021 14:44:49 GMT
• Title: Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
• Authors: Yanqiao Zhu, Yichen Xu, Hejie Cui, Carl Yang, Qiang Liu, Shu Wu
• Abstract summary: This work investigates Contrastive Learning (CL) on Graph Neural Networks (GNNs) We first generate multiple semantic views according to metapaths and network schemas. We then push node embeddings corresponding to different semantic views close to each other (positives) and pulling other embeddings apart (negatives) Considering the complex graph structure and the smoothing nature of GNNs, we propose a structure-aware hard negative mining scheme.
• Score: 21.702342154458623
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, heterogeneous Graph Neural Networks (GNNs) have become a de facto model for analyzing HGs, while most of them rely on a relative large number of labeled data. In this work, we investigate Contrastive Learning (CL), a key component in self-supervised approaches, on HGs to alleviate the label scarcity problem. We first generate multiple semantic views according to metapaths and network schemas. Then, by pushing node embeddings corresponding to different semantic views close to each other (positives) and pulling other embeddings apart (negatives), one can obtain informative representations without human annotations. However, this CL approach ignores the relative hardness of negative samples, which may lead to suboptimal performance. Considering the complex graph structure and the smoothing nature of GNNs, we propose a structure-aware hard negative mining scheme that measures hardness by structural characteristics for HGs. By synthesizing more negative nodes, we give larger weights to harder negatives with limited computational overhead to further boost the performance. Empirical studies on three real-world datasets show the effectiveness of our proposed method. The proposed method consistently outperforms existing state-of-the-art methods and notably, even surpasses several supervised counterparts.

Related papers

• Bootstrap Latents of Nodes and Neighbors for Graph Self-Supervised Learning [27.278097015083343]
  Contrastive learning requires negative samples to prevent model collapse and learn discriminative representations. We introduce a cross-attention module to predict the supportiveness score of a neighbor with respect to the anchor node. Our method mitigates class collision from negative and noisy positive samples, concurrently enhancing intra-class compactness.
  arXiv  Detail & Related papers  (2024-08-09T14:17:52Z)
• A GAN Approach for Node Embedding in Heterogeneous Graphs Using Subgraph Sampling [33.50085646298074]
  We propose a novel framework that combines Graph Neural Network (GNN) and Generative Adrial Network (GAN) to enhance classification for underrepresented node classes. The framework incorporates an advanced edge generation and selection module, enabling the simultaneous creation of synthetic nodes and edges.
  arXiv  Detail & Related papers  (2023-12-11T16:52:20Z)
• Mitigating Semantic Confusion from Hostile Neighborhood for Graph Active Learning [38.5372139056485]
  Graph Active Learning (GAL) aims to find the most informative nodes in graphs for annotation to maximize the Graph Neural Networks (GNNs) performance. Gal strategies may introduce semantic confusion to the selected training set, particularly when graphs are noisy. We present Semantic-aware Active learning framework for Graphs (SAG) to mitigate the semantic confusion problem.
  arXiv  Detail & Related papers  (2023-08-17T07:06:54Z)
• GCNH: A Simple Method For Representation Learning On Heterophilous Graphs [4.051099980410583]
  Graph Neural Networks (GNNs) are well-suited for learning on homophilous graphs. Recent works have proposed extensions to standard GNN architectures to improve performance on heterophilous graphs. We propose GCN for Heterophily (GCNH), a simple yet effective GNN architecture applicable to both heterophilous and homophilous scenarios.
  arXiv  Detail & Related papers  (2023-04-21T11:26:24Z)
• STERLING: Synergistic Representation Learning on Bipartite Graphs [78.86064828220613]
  A fundamental challenge of bipartite graph representation learning is how to extract node embeddings. Most recent bipartite graph SSL methods are based on contrastive learning which learns embeddings by discriminating positive and negative node pairs. We introduce a novel synergistic representation learning model (STERLING) to learn node embeddings without negative node pairs.
  arXiv  Detail & Related papers  (2023-01-25T03:21:42Z)
• Localized Contrastive Learning on Graphs [110.54606263711385]
  We introduce a simple yet effective contrastive model named Localized Graph Contrastive Learning (Local-GCL) In spite of its simplicity, Local-GCL achieves quite competitive performance in self-supervised node representation learning tasks on graphs with various scales and properties.
  arXiv  Detail & Related papers  (2022-12-08T23:36:00Z)
• Resisting Graph Adversarial Attack via Cooperative Homophilous Augmentation [60.50994154879244]
  Recent studies show that Graph Neural Networks are vulnerable and easily fooled by small perturbations. In this work, we focus on the emerging but critical attack, namely, Graph Injection Attack. We propose a general defense framework CHAGNN against GIA through cooperative homophilous augmentation of graph data and model.
  arXiv  Detail & Related papers  (2022-11-15T11:44:31Z)
• Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning [102.9138736545956]
  Heterogeneous graph neural network (HGNN) is a very popular technique for the modeling and analysis of heterogeneous graphs. We develop for the first time a novel and robust heterogeneous graph contrastive learning approach, namely HGCL, which introduces two views on respective guidance of node attributes and graph topologies. In this new approach, we adopt distinct but most suitable attribute and topology fusion mechanisms in the two views, which are conducive to mining relevant information in attributes and topologies separately.
  arXiv  Detail & Related papers  (2022-04-30T12:57:02Z)
• Tackling Oversmoothing of GNNs with Contrastive Learning [35.88575306925201]
  Graph neural networks (GNNs) integrate the comprehensive relation of graph data and representation learning capability. Oversmoothing makes the final representations of nodes indiscriminative, thus deteriorating the node classification and link prediction performance. We propose the Topology-guided Graph Contrastive Layer, named TGCL, which is the first de-oversmoothing method maintaining all three mentioned metrics.
  arXiv  Detail & Related papers  (2021-10-26T15:56:16Z)
• Uniting Heterogeneity, Inductiveness, and Efficiency for Graph Representation Learning [68.97378785686723]
  graph neural networks (GNNs) have greatly advanced the performance of node representation learning on graphs. A majority class of GNNs are only designed for homogeneous graphs, leading to inferior adaptivity to the more informative heterogeneous graphs. We propose a novel inductive, meta path-free message passing scheme that packs up heterogeneous node features with their associated edges from both low- and high-order neighbor nodes.
  arXiv  Detail & Related papers  (2021-04-04T23:31:39Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.