Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition

• URL: http://arxiv.org/abs/2504.12011v1
• Date: Wed, 16 Apr 2025 12:09:56 GMT
• Title: Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition
• Authors: Heesoo Jung, Hogun Park,
• Abstract summary: Self-supervised learning (SSL) in graphs has garnered significant attention, particularly in employing Graph Neural Networks (GNNs) with pretext tasks.<n>We show that existing methods position opposite ends of a spectrum driven by the graph embedding smoothness, with each end corresponding to outperformance on specific downstream tasks.<n>A framework, BSG, introduces novel loss functions designed to supplement the representation quality in graph-based SSL.
• Score: 3.330229314824913
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Self-supervised learning (SSL) in graphs has garnered significant attention, particularly in employing Graph Neural Networks (GNNs) with pretext tasks initially designed for other domains, such as contrastive learning and feature reconstruction. However, it remains uncertain whether these methods effectively reflect essential graph properties, precisely representation similarity with its neighbors. We observe that existing methods position opposite ends of a spectrum driven by the graph embedding smoothness, with each end corresponding to outperformance on specific downstream tasks. Decomposing the SSL objective into three terms via an information-theoretic framework with a neighbor representation variable reveals that this polarization stems from an imbalance among the terms, which existing methods may not effectively maintain. Further insights suggest that balancing between the extremes can lead to improved performance across a wider range of downstream tasks. A framework, BSG (Balancing Smoothness in Graph SSL), introduces novel loss functions designed to supplement the representation quality in graph-based SSL by balancing the derived three terms: neighbor loss, minimal loss, and divergence loss. We present a theoretical analysis of the effects of these loss functions, highlighting their significance from both the SSL and graph smoothness perspectives. Extensive experiments on multiple real-world datasets across node classification and link prediction consistently demonstrate that BSG achieves state-of-the-art performance, outperforming existing methods. Our implementation code is available at https://github.com/steve30572/BSG.

Related papers

• Oversmoothing as Loss of Sign: Towards Structural Balance in Graph Neural Networks [54.62268052283014]
  Oversmoothing is a common issue in graph neural networks (GNNs)<n>Three major classes of anti-oversmoothing techniques can be mathematically interpreted as message passing over signed graphs.<n>Negative edges can repel nodes to a certain extent, providing deeper insights into how these methods mitigate oversmoothing.
  arXiv  Detail & Related papers  (2025-02-17T03:25:36Z)
• Leveraging Joint Predictive Embedding and Bayesian Inference in Graph Self Supervised Learning [0.0]
  Graph representation learning has emerged as a cornerstone for tasks like node classification and link prediction.<n>Current self-supervised learning (SSL) methods face challenges such as computational inefficiency, reliance on contrastive objectives, and representation collapse.<n>We propose a novel joint embedding predictive framework for graph SSL that eliminates contrastive objectives and negative sampling while preserving semantic and structural information.
  arXiv  Detail & Related papers  (2025-02-02T07:42:45Z)
• Smoothness Really Matters: A Simple Yet Effective Approach for Unsupervised Graph Domain Adaptation [28.214010408550394]
  Unsupervised Graph Domain Adaptation (UGDA) seeks to bridge distribution shifts between domains by transferring knowledge from labeled source graphs to given unlabeled target graphs. We introduce a novel approach for UGDA called Target-Domain Structural Smoothing ( TDSS) TDSS is a simple and effective method designed to perform structural smoothing directly on the target graph.
  arXiv  Detail & Related papers  (2024-12-16T10:56:58Z)
• Point Cloud Denoising With Fine-Granularity Dynamic Graph Convolutional Networks [58.050130177241186]
  Noise perturbations often corrupt 3-D point clouds, hindering downstream tasks such as surface reconstruction, rendering, and further processing. This paper introduces finegranularity dynamic graph convolutional networks called GDGCN, a novel approach to denoising in 3-D point clouds.
  arXiv  Detail & Related papers  (2024-11-21T14:19:32Z)
• Conditional Distribution Learning on Graphs [15.730933577970687]
  We propose a conditional distribution learning (CDL) method that learns graph representations from graph-structured data for semisupervised graph classification.<n>Specifically, we present an end-to-end graph representation learning model to align the conditional distributions of weakly and strongly augmented features over the original features.
  arXiv  Detail & Related papers  (2024-11-20T07:26:36Z)
• VIGraph: Generative Self-supervised Learning for Class-Imbalanced Node Classification [9.686218058331061]
  Class imbalance in graph data presents significant challenges for node classification. Existing methods, such as SMOTE-based approaches, exhibit limitations in constructing imbalanced graphs. We introduce VIGraph, a simple yet effective generative SSL approach that relies on the Variational GAE as the fundamental model.
  arXiv  Detail & Related papers  (2023-11-02T12:36:19Z)
• Localized Contrastive Learning on Graphs [110.54606263711385]
  We introduce a simple yet effective contrastive model named Localized Graph Contrastive Learning (Local-GCL) In spite of its simplicity, Local-GCL achieves quite competitive performance in self-supervised node representation learning tasks on graphs with various scales and properties.
  arXiv  Detail & Related papers  (2022-12-08T23:36:00Z)
• Optimal Propagation for Graph Neural Networks [51.08426265813481]
  We propose a bi-level optimization approach for learning the optimal graph structure. We also explore a low-rank approximation model for further reducing the time complexity.
  arXiv  Detail & Related papers  (2022-05-06T03:37:00Z)
• Graph Structure Learning with Variational Information Bottleneck [70.62851953251253]
  We propose a novel Variational Information Bottleneck guided Graph Structure Learning framework, namely VIB-GSL. VIB-GSL learns an informative and compressive graph structure to distill the actionable information for specific downstream tasks.
  arXiv  Detail & Related papers  (2021-12-16T14:22:13Z)
• Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learning [64.98816284854067]
  Graph-based Semi-Supervised Learning (SSL) aims to transfer the labels of a handful of labeled data to the remaining massive unlabeled data via a graph. A novel GCN-based SSL algorithm is presented in this paper to enrich the supervision signals by utilizing both data similarities and graph structure.
  arXiv  Detail & Related papers  (2020-09-15T13:59:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.