Latent Augmentation For Better Graph Self-Supervised Learning

• URL: http://arxiv.org/abs/2206.12933v1
• Date: Sun, 26 Jun 2022 17:41:59 GMT
• Title: Latent Augmentation For Better Graph Self-Supervised Learning
• Authors: Jiashun Cheng, Man Li, Jia Li, Fugee Tsung
• Abstract summary: We argue that predictive models weaponed with latent augmentations and powerful decoder could achieve comparable or even better representation power than contrastive models. A novel graph decoder named Wiener Graph Deconvolutional Network is correspondingly designed to perform information reconstruction from augmented latent representations.
• Score: 20.082614919182692
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph self-supervised learning has been vastly employed to learn representations from unlabeled graphs. Existing methods can be roughly divided into predictive learning and contrastive learning, where the latter one attracts more research attention with better empirical performance. We argue that, however, predictive models weaponed with latent augmentations and powerful decoder could achieve comparable or even better representation power than contrastive models. In this work, we introduce data augmentations into latent space for superior generalization and better efficiency. A novel graph decoder named Wiener Graph Deconvolutional Network is correspondingly designed to perform information reconstruction from augmented latent representations. Theoretical analysis proves the superior reconstruction ability of graph wiener filter. Extensive experimental results on various datasets demonstrate the effectiveness of our approach.

Related papers

• Uncovering Capabilities of Model Pruning in Graph Contrastive Learning [0.0]
  We reformulate the problem of graph contrastive learning via contrasting different model versions rather than augmented views. We extensively validate our method on various benchmarks regarding graph classification via unsupervised and transfer learning.
  arXiv  Detail & Related papers  (2024-10-27T07:09:31Z)
• Spectral Augmentations for Graph Contrastive Learning [50.149996923976836]
  Contrastive learning has emerged as a premier method for learning representations with or without supervision. Recent studies have shown its utility in graph representation learning for pre-training. We propose a set of well-motivated graph transformation operations to provide a bank of candidates when constructing augmentations for a graph contrastive objective.
  arXiv  Detail & Related papers  (2023-02-06T16:26:29Z)
• Robust Causal Graph Representation Learning against Confounding Effects [21.380907101361643]
  We propose Robust Causal Graph Representation Learning (RCGRL) to learn robust graph representations against confounding effects. RCGRL introduces an active approach to generate instrumental variables under unconditional moment restrictions, which empowers the graph representation learning model to eliminate confounders.
  arXiv  Detail & Related papers  (2022-08-18T01:31:25Z)
• ARIEL: Adversarial Graph Contrastive Learning [51.14695794459399]
  ARIEL consistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks. ARIEL is more robust in the face of adversarial attacks.
  arXiv  Detail & Related papers  (2022-08-15T01:24:42Z)
• Adversarial Graph Contrastive Learning with Information Regularization [51.14695794459399]
  Contrastive learning is an effective method in graph representation learning. Data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples. We propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL) It consistently outperforms the current graph contrastive learning methods in the node classification task over various real-world datasets.
  arXiv  Detail & Related papers  (2022-02-14T05:54:48Z)
• Bootstrapping Informative Graph Augmentation via A Meta Learning Approach [21.814940639910358]
  In graph contrastive learning, benchmark methods apply various graph augmentation approaches. Most of the augmentation methods are non-learnable, which causes the issue of generating unbeneficial augmented graphs. We motivate our method to generate augmented graph by a learnable graph augmenter, called MEta Graph Augmentation (MEGA)
  arXiv  Detail & Related papers  (2022-01-11T07:15:13Z)
• Graph Representation Learning by Ensemble Aggregating Subgraphs via Mutual Information Maximization [5.419711903307341]
  We introduce a self-supervised learning method to enhance the representations of graph-level learned by Graph Neural Networks. To get a comprehensive understanding of the graph structure, we propose an ensemble-learning like subgraph method. And to achieve efficient and effective contrasive learning, a Head-Tail contrastive samples construction method is proposed.
  arXiv  Detail & Related papers  (2021-03-24T12:06:12Z)
• Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification [12.423303337249795]
  We study the phenomenon of catastrophic forgetting in the graph representation learning scenario. We find that replay is the most effective strategy in so far, which also benefits the most from the use of regularization.
  arXiv  Detail & Related papers  (2021-03-22T12:07:21Z)
• Iterative Graph Self-Distillation [161.04351580382078]
  We propose a novel unsupervised graph learning paradigm called Iterative Graph Self-Distillation (IGSD) IGSD iteratively performs the teacher-student distillation with graph augmentations. We show that we achieve significant and consistent performance gain on various graph datasets in both unsupervised and semi-supervised settings.
  arXiv  Detail & Related papers  (2020-10-23T18:37:06Z)
• Graph Contrastive Learning with Augmentations [109.23158429991298]
  We propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We show that our framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-22T20:13:43Z)
• Towards Deeper Graph Neural Networks [63.46470695525957]
  Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Several recent studies attribute this performance deterioration to the over-smoothing issue. We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
  arXiv  Detail & Related papers  (2020-07-18T01:11:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.