PyG-SSL: A Graph Self-Supervised Learning Toolkit

• URL: http://arxiv.org/abs/2412.21151v1
• Date: Mon, 30 Dec 2024 18:32:05 GMT
• Title: PyG-SSL: A Graph Self-Supervised Learning Toolkit
• Authors: Lecheng Zheng, Baoyu Jing, Zihao Li, Zhichen Zeng, Tianxin Wei, Mengting Ai, Xinrui He, Lihui Liu, Dongqi Fu, Jiaxuan You, Hanghang Tong, Jingrui He,
• Abstract summary: Graph Self-Supervised Learning (SSL) has emerged as a pivotal area of research in recent years. Despite the remarkable achievements of these graph SSL methods, their current implementation poses significant challenges for beginners. We present a Graph SSL toolkit named PyG-SSL, which is built upon PyTorch and is compatible with various deep learning and scientific computing backends.
• Score: 71.22547762704602
• License:
• Abstract: Graph Self-Supervised Learning (SSL) has emerged as a pivotal area of research in recent years. By engaging in pretext tasks to learn the intricate topological structures and properties of graphs using unlabeled data, these graph SSL models achieve enhanced performance, improved generalization, and heightened robustness. Despite the remarkable achievements of these graph SSL methods, their current implementation poses significant challenges for beginners and practitioners due to the complex nature of graph structures, inconsistent evaluation metrics, and concerns regarding reproducibility hinder further progress in this field. Recognizing the growing interest within the research community, there is an urgent need for a comprehensive, beginner-friendly, and accessible toolkit consisting of the most representative graph SSL algorithms. To address these challenges, we present a Graph SSL toolkit named PyG-SSL, which is built upon PyTorch and is compatible with various deep learning and scientific computing backends. Within the toolkit, we offer a unified framework encompassing dataset loading, hyper-parameter configuration, model training, and comprehensive performance evaluation for diverse downstream tasks. Moreover, we provide beginner-friendly tutorials and the best hyper-parameters of each graph SSL algorithm on different graph datasets, facilitating the reproduction of results. The GitHub repository of the library is https://github.com/iDEA-iSAIL-Lab-UIUC/pyg-ssl.

Related papers

• Do Neural Scaling Laws Exist on Graph Self-Supervised Learning? [9.297227372861876]
  Self-supervised learning(SSL) is essential to obtain foundation models in NLP and CV domains via effectively leveraging knowledge in large-scale unlabeled data. It remains a mystery whether existing SSL in the graph domain can follow the scaling behavior toward building Graph Foundation Models(GFMs) with large-scale pre-training. This paper examines existing SSL techniques for the feasibility of Graph SSL techniques in developing GFMs and opens a new direction for graph SSL design with the new evaluation prototype.
  arXiv  Detail & Related papers  (2024-08-20T23:45:11Z)
• Towards Graph Contrastive Learning: A Survey and Beyond [23.109430624817637]
  Self-supervised learning (SSL) on graphs has gained increasing attention and has made significant progress. SSL enables machine learning models to produce informative representations from unlabeled graph data. Graph Contrastive Learning (GCL) has not been thoroughly investigated in the existing literature.
  arXiv  Detail & Related papers  (2024-05-20T08:19:10Z)
• Continual Learning on Graphs: Challenges, Solutions, and Opportunities [72.7886669278433]
  We provide a comprehensive review of existing continual graph learning (CGL) algorithms. We compare methods with traditional continual learning techniques and analyze the applicability of the traditional continual learning techniques to forgetting tasks. We will maintain an up-to-date repository featuring a comprehensive list of accessible algorithms.
  arXiv  Detail & Related papers  (2024-02-18T12:24:45Z)
• GSLB: The Graph Structure Learning Benchmark [34.859275408785614]
  Graph Structure Learning (GSL) has recently garnered considerable attention due to its ability to optimize both the parameters of Graph Neural Networks (GNNs) and the computation graph structure simultaneously. There is no standard experimental setting or fair comparison for performance evaluation, which creates a great obstacle to understanding the progress in this field. We develop a comprehensive Graph Structure Learning Benchmark (GSLB) curated from 20 diverse graph datasets and 16 distinct GSL algorithms.
  arXiv  Detail & Related papers  (2023-10-08T14:13:03Z)
• GraphMAE: Self-Supervised Masked Graph Autoencoders [52.06140191214428]
  We present a masked graph autoencoder GraphMAE that mitigates issues for generative self-supervised graph learning. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE--a simple graph autoencoder with our careful designs--can consistently generate outperformance over both contrastive and generative state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-05-22T11:57:08Z)
• Self-supervised on Graphs: Contrastive, Generative,or Predictive [25.679620842010422]
  Self-supervised learning (SSL) is emerging as a new paradigm for extracting informative knowledge through well-designed pretext tasks. We divide existing graph SSL methods into three categories: contrastive, generative, and predictive. We also summarize the commonly used datasets, evaluation metrics, downstream tasks, and open-source implementations of various algorithms.
  arXiv  Detail & Related papers  (2021-05-16T03:30:03Z)
• Graph Self-Supervised Learning: A Survey [73.86209411547183]
  Self-supervised learning (SSL) has become a promising and trending learning paradigm for graph data. We present a timely and comprehensive review of the existing approaches which employ SSL techniques for graph data.
  arXiv  Detail & Related papers  (2021-02-27T03:04:21Z)
• Graph-based Semi-supervised Learning: A Comprehensive Review [51.26862262550445]
  Semi-supervised learning (SSL) has tremendous value in practice due to its ability to utilize both labeled data and unlabelled data. An important class of SSL methods is to naturally represent data as graphs, which corresponds to graph-based semi-supervised learning (GSSL) methods. GSSL methods have demonstrated their advantages in various domains due to their uniqueness of structure, the universality of applications, and their scalability to large scale data.
  arXiv  Detail & Related papers  (2021-02-26T05:11:09Z)
• Self-supervised Learning on Graphs: Deep Insights and New Direction [66.78374374440467]
  Self-supervised learning (SSL) aims to create domain specific pretext tasks on unlabeled data. There are increasing interests in generalizing deep learning to the graph domain in the form of graph neural networks (GNNs)
  arXiv  Detail & Related papers  (2020-06-17T20:30:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.