Preserving Node Distinctness in Graph Autoencoders via Similarity Distillation

• URL: http://arxiv.org/abs/2406.17517v1
• Date: Tue, 25 Jun 2024 12:54:35 GMT
• Title: Preserving Node Distinctness in Graph Autoencoders via Similarity Distillation
• Authors: Ge Chen, Yulan Hu, Sheng Ouyang, Yong Liu, Cuicui Luo,
• Abstract summary: Graph autoencoders (GAEs) rely on distance-based criteria, such as mean-square-error (MSE) to reconstruct the input graph. relying solely on a single reconstruction criterion may lead to a loss of distinctiveness in the reconstructed graph. We have developed a simple yet effective strategy to preserve the necessary distinctness in the reconstructed graph.
• Score: 9.395697548237333
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Graph autoencoders (GAEs), as a kind of generative self-supervised learning approach, have shown great potential in recent years. GAEs typically rely on distance-based criteria, such as mean-square-error (MSE), to reconstruct the input graph. However, relying solely on a single reconstruction criterion may lead to a loss of distinctiveness in the reconstructed graph, causing nodes to collapse into similar representations and resulting in sub-optimal performance. To address this issue, we have developed a simple yet effective strategy to preserve the necessary distinctness in the reconstructed graph. Inspired by the knowledge distillation technique, we found that the dual encoder-decoder architecture of GAEs can be viewed as a teacher-student relationship. Therefore, we propose transferring the knowledge of distinctness from the raw graph to the reconstructed graph, achieved through a simple KL constraint. Specifically, we compute pairwise node similarity scores in the raw graph and reconstructed graph. During the training process, the KL constraint is optimized alongside the reconstruction criterion. We conducted extensive experiments across three types of graph tasks, demonstrating the effectiveness and generality of our strategy. This indicates that the proposed approach can be employed as a plug-and-play method to avoid vague reconstructions and enhance overall performance.

Related papers

• GraphCroc: Cross-Correlation Autoencoder for Graph Structural Reconstruction [6.817416560637197]
  Graph autoencoders (GAEs) reconstruct graph structures from node embeddings. We introduce a cross-correlation mechanism that significantly enhances the GAE representational capabilities. We also propose GraphCroc, a new GAE that supports flexible encoder architectures tailored for various downstream tasks.
  arXiv  Detail & Related papers  (2024-10-04T12:59:45Z)
• HC-GAE: The Hierarchical Cluster-based Graph Auto-Encoder for Graph Representation Learning [24.641827220223682]
  We develop a novel Hierarchical Cluster-based GAE (HC-GAE) that can learn effective structural characteristics for graph data analysis. The proposed HC-GAE can generate effective representations for either node classification or graph classification, and the experiments demonstrate the effectiveness on real-world datasets.
  arXiv  Detail & Related papers  (2024-05-23T16:08:04Z)
• Rethinking Graph Masked Autoencoders through Alignment and Uniformity [26.86368034133612]
  Self-supervised learning on graphs can be bifurcated into contrastive and generative methods. Recent advent of graph masked autoencoder (GraphMAE) rekindles momentum behind generative methods.
  arXiv  Detail & Related papers  (2024-02-11T15:21:08Z)
• Localized Contrastive Learning on Graphs [110.54606263711385]
  We introduce a simple yet effective contrastive model named Localized Graph Contrastive Learning (Local-GCL) In spite of its simplicity, Local-GCL achieves quite competitive performance in self-supervised node representation learning tasks on graphs with various scales and properties.
  arXiv  Detail & Related papers  (2022-12-08T23:36:00Z)
• Graph Contrastive Learning with Implicit Augmentations [36.57536688367965]
  Implicit Graph Contrastive Learning (iGCL) uses augmentations in latent space learned from a Variational Graph Auto-Encoder by reconstructing graph topological structure. Experimental results on both graph-level and node-level tasks show that the proposed method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-11-07T17:34:07Z)
• GraphCoCo: Graph Complementary Contrastive Learning [65.89743197355722]
  Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. This paper proposes an effective graph complementary contrastive learning approach named GraphCoCo to tackle the above issue.
  arXiv  Detail & Related papers  (2022-03-24T02:58:36Z)
• Towards Unsupervised Deep Graph Structure Learning [67.58720734177325]
  We propose an unsupervised graph structure learning paradigm, where the learned graph topology is optimized by data itself without any external guidance. Specifically, we generate a learning target from the original data as an "anchor graph", and use a contrastive loss to maximize the agreement between the anchor graph and the learned graph.
  arXiv  Detail & Related papers  (2022-01-17T11:57:29Z)
• A Robust and Generalized Framework for Adversarial Graph Embedding [73.37228022428663]
  We propose a robust framework for adversarial graph embedding, named AGE. AGE generates the fake neighbor nodes as the enhanced negative samples from the implicit distribution. Based on this framework, we propose three models to handle three types of graph data.
  arXiv  Detail & Related papers  (2021-05-22T07:05:48Z)
• Diversified Multiscale Graph Learning with Graph Self-Correction [55.43696999424127]
  We propose a diversified multiscale graph learning model equipped with two core ingredients. A graph self-correction (GSC) mechanism to generate informative embedded graphs, and a diversity boosting regularizer (DBR) to achieve a comprehensive characterization of the input graph. Experiments on popular graph classification benchmarks show that the proposed GSC mechanism leads to significant improvements over state-of-the-art graph pooling methods.
  arXiv  Detail & Related papers  (2021-03-17T16:22:24Z)
• Self-Constructing Graph Convolutional Networks for Semantic Labeling [23.623276007011373]
  We propose a novel architecture called the Self-Constructing Graph (SCG), which makes use of learnable latent variables to generate embeddings. SCG can automatically obtain optimized non-local context graphs from complex-shaped objects in aerial imagery. We demonstrate the effectiveness and flexibility of the proposed SCG on the publicly available ISPRS Vaihingen dataset.
  arXiv  Detail & Related papers  (2020-03-15T21:55:24Z)
• Embedding Graph Auto-Encoder for Graph Clustering [90.8576971748142]
  Graph auto-encoder (GAE) models are based on semi-supervised graph convolution networks (GCN) We design a specific GAE-based model for graph clustering to be consistent with the theory, namely Embedding Graph Auto-Encoder (EGAE) EGAE consists of one encoder and dual decoders.
  arXiv  Detail & Related papers  (2020-02-20T09:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.