Related papers: COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning

COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning

URL: http://arxiv.org/abs/2206.04726v2
Date: Mon, 13 Jun 2022 07:11:26 GMT
Title: COSTA: Covariance-Preserving Feature Augmentation for Graph Contrastive Learning
Authors: Yifei Zhang and Hao Zhu and Zixing Song and Piotr Koniusz and Irwin King
Abstract summary: We show that the node embedding obtained via the graph augmentations is highly biased. Instead of investigating graph augmentation in the input space, we propose augmentations on the hidden features. We show that the feature augmentation with COSTA achieves comparable/better results than graph augmentation based models.
Score: 64.78221638149276
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Graph contrastive learning (GCL) improves graph representation learning, leading to SOTA on various downstream tasks. The graph augmentation step is a vital but scarcely studied step of GCL. In this paper, we show that the node embedding obtained via the graph augmentations is highly biased, somewhat limiting contrastive models from learning discriminative features for downstream tasks. Thus, instead of investigating graph augmentation in the input space, we alternatively propose to perform augmentations on the hidden features (feature augmentation). Inspired by so-called matrix sketching, we propose COSTA, a novel COvariance-preServing feaTure space Augmentation framework for GCL, which generates augmented features by maintaining a "good sketch" of original features. To highlight the superiority of feature augmentation with COSTA, we investigate a single-view setting (in addition to multi-view one) which conserves memory and computations. We show that the feature augmentation with COSTA achieves comparable/better results than graph augmentation based models.

Related papers

Community-Invariant Graph Contrastive Learning [21.72222875193335]
This research investigates the role of the graph community in graph augmentation. We propose a community-invariant GCL framework to maintain graph community structure during learnable graph augmentation.
arXiv Detail & Related papers (2024-05-02T14:59:58Z)
Hybrid Augmented Automated Graph Contrastive Learning [3.785553471764994]
We propose a framework called Hybrid Augmented Automated Graph Contrastive Learning (HAGCL) HAGCL consists of a feature-level learnable view generator and an edge-level learnable view generator. It insures to learn the most semantically meaningful structure in terms of features and topology.
arXiv Detail & Related papers (2023-03-24T03:26:20Z)
Features Based Adaptive Augmentation for Graph Contrastive Learning [0.0]
Self-Supervised learning aims to eliminate the need for expensive annotation in graph representation learning. We introduce a Feature Based Adaptive Augmentation (FebAA) approach, which identifies and preserves potentially influential features. We successfully improved the accuracy of GRACE and BGRL on eight graph representation learning's benchmark datasets.
arXiv Detail & Related papers (2022-07-05T03:41:20Z)
GraphCoCo: Graph Complementary Contrastive Learning [65.89743197355722]
Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. This paper proposes an effective graph complementary contrastive learning approach named GraphCoCo to tackle the above issue.
arXiv Detail & Related papers (2022-03-24T02:58:36Z)
Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations [94.41860307845812]
Self-supervision is recently surging at its new frontier of graph learning. GraphCL uses a prefabricated prior reflected by the ad-hoc manual selection of graph data augmentations. We have extended the prefabricated discrete prior in the augmentation set, to a learnable continuous prior in the parameter space of graph generators. We have leveraged both principles of information minimization (InfoMin) and information bottleneck (InfoBN) to regularize the learned priors.
arXiv Detail & Related papers (2022-01-04T15:49:18Z)
Augmentation-Free Self-Supervised Learning on Graphs [7.146027549101716]
We propose a novel augmentation-free self-supervised learning framework for graphs, named AFGRL. Specifically, we generate an alternative view of a graph by discovering nodes that share the local structural information and the global semantics with the graph.
arXiv Detail & Related papers (2021-12-05T04:20:44Z)
Graph Contrastive Learning Automated [94.41860307845812]
Graph contrastive learning (GraphCL) has emerged with promising representation learning performance. The effectiveness of GraphCL hinges on ad-hoc data augmentations, which have to be manually picked per dataset. This paper proposes a unified bi-level optimization framework to automatically, adaptively and dynamically select data augmentations when performing GraphCL on specific graph data.
arXiv Detail & Related papers (2021-06-10T16:35:27Z)
Graph Contrastive Learning with Augmentations [109.23158429991298]
We propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We show that our framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods.
arXiv Detail & Related papers (2020-10-22T20:13:43Z)
Robust Optimization as Data Augmentation for Large-scale Graphs [117.2376815614148]
We propose FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training. FLAG is a general-purpose approach for graph data, which universally works in node classification, link prediction, and graph classification tasks.
arXiv Detail & Related papers (2020-10-19T21:51:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.