ExGRG: Explicitly-Generated Relation Graph for Self-Supervised Representation Learning

• URL: http://arxiv.org/abs/2402.06737v2
• Date: Tue, 4 Jun 2024 15:30:15 GMT
• Title: ExGRG: Explicitly-Generated Relation Graph for Self-Supervised Representation Learning
• Authors: Mahdi Naseri, Mahdi Biparva,
• Abstract summary: Self-supervised learning has emerged as a powerful technique in pre-training deep learning models. This paper introduces a novel non-contrastive SSL approach to Explicitly Generate a compositional Relation Graph.
• Score: 4.105236597768038
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-supervised Learning (SSL) has emerged as a powerful technique in pre-training deep learning models without relying on expensive annotated labels, instead leveraging embedded signals in unlabeled data. While SSL has shown remarkable success in computer vision tasks through intuitive data augmentation, its application to graph-structured data poses challenges due to the semantic-altering and counter-intuitive nature of graph augmentations. Addressing this limitation, this paper introduces a novel non-contrastive SSL approach to Explicitly Generate a compositional Relation Graph (ExGRG) instead of relying solely on the conventional augmentation-based implicit relation graph. ExGRG offers a framework for incorporating prior domain knowledge and online extracted information into the SSL invariance objective, drawing inspiration from the Laplacian Eigenmap and Expectation-Maximization (EM). Employing an EM perspective on SSL, our E-step involves relation graph generation to identify candidates to guide the SSL invariance objective, and M-step updates the model parameters by integrating the derived relational information. Extensive experimentation on diverse node classification datasets demonstrates the superiority of our method over state-of-the-art techniques, affirming ExGRG as an effective adoption of SSL for graph representation learning.

Related papers

• Self-Supervised Conditional Distribution Learning on Graphs [15.730933577970687]
  We present an end-to-end graph representation learning model to align the conditional distributions of weakly and strongly augmented features over the original features. This alignment effectively reduces the risk of disrupting intrinsic semantic information through graph-structured data augmentation.
  arXiv  Detail & Related papers  (2024-11-20T07:26:36Z)
• CONVERT:Contrastive Graph Clustering with Reliable Augmentation [110.46658439733106]
  We propose a novel CONtrastiVe Graph ClustEring network with Reliable AugmenTation (CONVERT) In our method, the data augmentations are processed by the proposed reversible perturb-recover network. To further guarantee the reliability of semantics, a novel semantic loss is presented to constrain the network.
  arXiv  Detail & Related papers  (2023-08-17T13:07:09Z)
• Analyzing Data-Centric Properties for Contrastive Learning on Graphs [32.69353929886551]
  We investigate how do graph SSL methods, such as contrastive learning (CL), work well? Our work rigorously contextualizes, both empirically and theoretically, the effects of data-centric properties on augmentation strategies and learning paradigms for graph SSL.
  arXiv  Detail & Related papers  (2022-08-04T17:58:37Z)
• GraphMAE: Self-Supervised Masked Graph Autoencoders [52.06140191214428]
  We present a masked graph autoencoder GraphMAE that mitigates issues for generative self-supervised graph learning. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE--a simple graph autoencoder with our careful designs--can consistently generate outperformance over both contrastive and generative state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-05-22T11:57:08Z)
• Towards Unsupervised Deep Graph Structure Learning [67.58720734177325]
  We propose an unsupervised graph structure learning paradigm, where the learned graph topology is optimized by data itself without any external guidance. Specifically, we generate a learning target from the original data as an "anchor graph", and use a contrastive loss to maximize the agreement between the anchor graph and the learned graph.
  arXiv  Detail & Related papers  (2022-01-17T11:57:29Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• Graph Contrastive Learning with Adaptive Augmentation [23.37786673825192]
  We propose a novel graph contrastive representation learning method with adaptive augmentation. Specifically, we design augmentation schemes based on node centrality measures to highlight important connective structures. Our proposed method consistently outperforms existing state-of-the-art baselines and even surpasses some supervised counterparts.
  arXiv  Detail & Related papers  (2020-10-27T15:12:21Z)
• Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning [64.98816284854067]
  Graph-based Semi-Supervised Learning (SSL) aims to transfer the labels of a handful of labeled data to the remaining massive unlabeled data via a graph. A novel GCN-based SSL algorithm is presented in this paper to enrich the supervision signals by utilizing both data similarities and graph structure.
  arXiv  Detail & Related papers  (2020-09-15T13:59:28Z)
• Tensor Graph Convolutional Networks for Multi-relational and Robust Learning [74.05478502080658]
  This paper introduces a tensor-graph convolutional network (TGCN) for scalable semi-supervised learning (SSL) from data associated with a collection of graphs, that are represented by a tensor. The proposed architecture achieves markedly improved performance relative to standard GCNs, copes with state-of-the-art adversarial attacks, and leads to remarkable SSL performance over protein-to-protein interaction networks.
  arXiv  Detail & Related papers  (2020-03-15T02:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.