Certifiably Robust Graph Contrastive Learning
- URL: http://arxiv.org/abs/2310.03312v1
- Date: Thu, 5 Oct 2023 05:00:11 GMT
- Title: Certifiably Robust Graph Contrastive Learning
- Authors: Minhua Lin, Teng Xiao, Enyan Dai, Xiang Zhang, Suhang Wang
- Abstract summary: We develop the first certifiably robust framework in Graph Contrastive Learning (GCL)
We first propose a unified criteria to evaluate and certify the robustness of GCL.
We then introduce a novel technique, RES (Randomized Edgedrop Smoothing), to ensure certifiable robustness for any GCL model.
- Score: 43.029361784095016
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Graph Contrastive Learning (GCL) has emerged as a popular unsupervised graph
representation learning method. However, it has been shown that GCL is
vulnerable to adversarial attacks on both the graph structure and node
attributes. Although empirical approaches have been proposed to enhance the
robustness of GCL, the certifiable robustness of GCL is still remain
unexplored. In this paper, we develop the first certifiably robust framework in
GCL. Specifically, we first propose a unified criteria to evaluate and certify
the robustness of GCL. We then introduce a novel technique, RES (Randomized
Edgedrop Smoothing), to ensure certifiable robustness for any GCL model, and
this certified robustness can be provably preserved in downstream tasks.
Furthermore, an effective training method is proposed for robust GCL. Extensive
experiments on real-world datasets demonstrate the effectiveness of our
proposed method in providing effective certifiable robustness and enhancing the
robustness of any GCL model. The source code of RES is available at
https://github.com/ventr1c/RES-GCL.
Related papers
- Graph-level Protein Representation Learning by Structure Knowledge
Refinement [50.775264276189695]
This paper focuses on learning representation on the whole graph level in an unsupervised manner.
We propose a novel framework called Structure Knowledge Refinement (SKR) which uses data structure to determine the probability of whether a pair is positive or negative.
arXiv Detail & Related papers (2024-01-05T09:05:33Z) - Rethinking and Simplifying Bootstrapped Graph Latents [48.76934123429186]
Graph contrastive learning (GCL) has emerged as a representative paradigm in graph self-supervised learning.
We present SGCL, a simple yet effective GCL framework that utilizes the outputs from two consecutive iterations as positive pairs.
We show that SGCL can achieve competitive performance with fewer parameters, lower time and space costs, and significant convergence speedup.
arXiv Detail & Related papers (2023-12-05T09:49:50Z) - On the Adversarial Robustness of Graph Contrastive Learning Methods [9.675856264585278]
We introduce a comprehensive evaluation robustness protocol tailored to assess the robustness of graph contrastive learning (GCL) models.
We subject these models to adaptive adversarial attacks targeting the graph structure, specifically in the evasion scenario.
With our work, we aim to offer insights into the robustness of GCL methods and hope to open avenues for potential future research directions.
arXiv Detail & Related papers (2023-11-29T17:59:18Z) - Architecture Matters: Uncovering Implicit Mechanisms in Graph
Contrastive Learning [34.566003077992384]
We present a systematic study of various graph contrastive learning (GCL) methods.
By uncovering how the implicit inductive bias of GNNs works in contrastive learning, we theoretically provide insights into the above intriguing properties of GCL.
Rather than directly porting existing NN methods to GCL, we advocate for more attention toward the unique architecture of graph learning.
arXiv Detail & Related papers (2023-11-05T15:54:17Z) - Homophily-Driven Sanitation View for Robust Graph Contrastive Learning [28.978770069310276]
We investigate adversarial robustness of unsupervised Graph Contrastive Learning (GCL) against structural attacks.
We present a robust GCL framework that integrates a homophily-driven sanitation view, which can be learned jointly with contrastive learning.
We conduct extensive experiments to evaluate the performance of our proposed model, GCHS, against two state of the art structural attacks on GCL.
arXiv Detail & Related papers (2023-07-24T06:41:59Z) - CSGCL: Community-Strength-Enhanced Graph Contrastive Learning [13.770188320382285]
We propose a Community-Strength-enhanced Graph Contrastive Learning (CSGCL) framework to preserve community strength throughout the learning process.
We present two novel graph augmentation methods, Communal Attribute Voting (CAV) and Communal Edge Dropping (CED), where the perturbations of node attributes and edges are guided by community strength.
We report extensive experiment results on three downstream tasks: node classification, node clustering, and link prediction.
arXiv Detail & Related papers (2023-05-08T12:21:24Z) - Single-Pass Contrastive Learning Can Work for Both Homophilic and
Heterophilic Graph [60.28340453547902]
Graph contrastive learning (GCL) techniques typically require two forward passes for a single instance to construct the contrastive loss.
Existing GCL approaches fail to provide strong performance guarantees.
We implement the Single-Pass Graph Contrastive Learning method (SP-GCL)
Empirically, the features learned by the SP-GCL can match or outperform existing strong baselines with significantly less computational overhead.
arXiv Detail & Related papers (2022-11-20T07:18:56Z) - Unifying Graph Contrastive Learning with Flexible Contextual Scopes [57.86762576319638]
We present a self-supervised learning method termed Unifying Graph Contrastive Learning with Flexible Contextual Scopes (UGCL for short)
Our algorithm builds flexible contextual representations with contextual scopes by controlling the power of an adjacency matrix.
Based on representations from both local and contextual scopes, distL optimises a very simple contrastive loss function for graph representation learning.
arXiv Detail & Related papers (2022-10-17T07:16:17Z) - Uncovering the Structural Fairness in Graph Contrastive Learning [87.65091052291544]
Graph contrastive learning (GCL) has emerged as a promising self-supervised approach for learning node representations.
We show that representations obtained by GCL methods are already fairer to degree bias than those learned by GCN.
We devise a novel graph augmentation method, called GRAph contrastive learning for DEgree bias (GRADE), which applies different strategies to low- and high-degree nodes.
arXiv Detail & Related papers (2022-10-06T15:58:25Z) - An Empirical Study of Graph Contrastive Learning [17.246488437677616]
Graph Contrastive Learning establishes a new paradigm for learning graph representations without human annotations.
We identify several critical design considerations within a general GCL paradigm, including augmentation functions, contrasting modes, contrastive objectives, and negative mining techniques.
To foster future research and ease the implementation of GCL algorithms, we develop an easy-to-use library PyGCL, featuring modularized CL components, standardized evaluation, and experiment management.
arXiv Detail & Related papers (2021-09-02T17:43:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.