Graph Learning Indexer: A Contributor-Friendly and Metadata-Rich Platform for Graph Learning Benchmarks

• URL: http://arxiv.org/abs/2212.04537v1
• Date: Thu, 8 Dec 2022 19:57:01 GMT
• Title: Graph Learning Indexer: A Contributor-Friendly and Metadata-Rich Platform for Graph Learning Benchmarks
• Authors: Jiaqi Ma, Xingjian Zhang, Hezheng Fan, Jin Huang, Tianyue Li, Ting Wei Li, Yiwen Tu, Chenshu Zhu, Qiaozhu Mei
• Abstract summary: Graph Learning Indexer (GLI) is a benchmark curation platform for graph learning. GLI is designed to incentivize emphdataset contributors. GLI curates a knowledge base, instead of a plain collection, of benchmark datasets.
• Score: 11.972121836128592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Establishing open and general benchmarks has been a critical driving force behind the success of modern machine learning techniques. As machine learning is being applied to broader domains and tasks, there is a need to establish richer and more diverse benchmarks to better reflect the reality of the application scenarios. Graph learning is an emerging field of machine learning that urgently needs more and better benchmarks. To accommodate the need, we introduce Graph Learning Indexer (GLI), a benchmark curation platform for graph learning. In comparison to existing graph learning benchmark libraries, GLI highlights two novel design objectives. First, GLI is designed to incentivize \emph{dataset contributors}. In particular, we incorporate various measures to minimize the effort of contributing and maintaining a dataset, increase the usability of the contributed dataset, as well as encourage attributions to different contributors of the dataset. Second, GLI is designed to curate a knowledge base, instead of a plain collection, of benchmark datasets. We use multiple sources of meta information to augment the benchmark datasets with \emph{rich characteristics}, so that they can be easily selected and used in downstream research or development. The source code of GLI is available at \url{https://github.com/Graph-Learning-Benchmarks/gli}.

Related papers

• How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension [53.6373473053431]
  This work introduces a benchmark to assess large language models' capabilities in graph pattern tasks. We have developed a benchmark that evaluates whether LLMs can understand graph patterns based on either terminological or topological descriptions. Our benchmark encompasses both synthetic and real datasets, and a variety of models, with a total of 11 tasks and 7 models.
  arXiv  Detail & Related papers  (2024-10-04T04:48:33Z)
• Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models [90.98855064914379]
  We introduce ProGraph, a benchmark for large language models (LLMs) to process graphs. Our findings reveal that the performance of current LLMs is unsatisfactory, with the best model achieving only 36% accuracy. We propose LLM4Graph datasets, which include crawled documents and auto-generated codes based on 6 widely used graph libraries.
  arXiv  Detail & Related papers  (2024-09-29T11:38:45Z)
• Large Language Models on Graphs: A Comprehensive Survey [77.16803297418201]
  We provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets.
  arXiv  Detail & Related papers  (2023-12-05T14:14:27Z)
• GraphGPT: Graph Instruction Tuning for Large Language Models [27.036935149004726]
  Graph Neural Networks (GNNs) have evolved to understand graph structures. To enhance robustness, self-supervised learning (SSL) has become a vital tool for data augmentation. Our research tackles this by advancing graph model generalization in zero-shot learning environments.
  arXiv  Detail & Related papers  (2023-10-19T06:17:46Z)
• SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [131.04781590452308]
  We present SimTeG, a frustratingly Simple approach for Textual Graph learning. We first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task. We then generate node embeddings using the last hidden states of finetuned LM.
  arXiv  Detail & Related papers  (2023-08-03T07:00:04Z)
• Curriculum Graph Machine Learning: A Survey [51.89783017927647]
  curriculum graph machine learning (Graph CL) integrates the strength of graph machine learning and curriculum learning. This paper comprehensively overview approaches on Graph CL and present a detailed survey of recent advances in this direction.
  arXiv  Detail & Related papers  (2023-02-06T16:59:25Z)
• A Framework for Large Scale Synthetic Graph Dataset Generation [2.248608623448951]
  This work proposes a scalable synthetic graph generation tool to scale the datasets to production-size graphs. The tool learns a series of parametric models from proprietary datasets that can be released to researchers. We demonstrate the generalizability of the framework across a series of datasets.
  arXiv  Detail & Related papers  (2022-10-04T22:41:33Z)
• Automated Graph Machine Learning: Approaches, Libraries, Benchmarks and Directions [58.220137936626315]
  This paper extensively discusses automated graph machine learning approaches. We introduce AutoGL, our dedicated and the world's first open-source library for automated graph machine learning. Also, we describe a tailored benchmark that supports unified, reproducible, and efficient evaluations.
  arXiv  Detail & Related papers  (2022-01-04T18:31:31Z)
• Graph Few-shot Class-incremental Learning [25.94168397283495]
  The ability to incrementally learn new classes is vital to all real-world artificial intelligence systems. In this paper, we investigate the challenging yet practical problem, Graph Few-shot Class-incremental (Graph FCL) problem. We put forward a Graph Pseudo Incremental Learning paradigm by sampling tasks recurrently from the base classes. We present a task-sensitive regularizer calculated from task-level attention and node class prototypes to mitigate overfitting onto either novel or base classes.
  arXiv  Detail & Related papers  (2021-12-23T19:46:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.