RelBench: A Benchmark for Deep Learning on Relational Databases

• URL: http://arxiv.org/abs/2407.20060v1
• Date: Mon, 29 Jul 2024 14:46:13 GMT
• Title: RelBench: A Benchmark for Deep Learning on Relational Databases
• Authors: Joshua Robinson, Rishabh Ranjan, Weihua Hu, Kexin Huang, Jiaqi Han, Alejandro Dobles, Matthias Fey, Jan E. Lenssen, Yiwen Yuan, Zecheng Zhang, Xinwei He, Jure Leskovec,
• Abstract summary: We present RelBench, a public benchmark for solving tasks over databases with graph neural networks. We use RelBench to conduct the first comprehensive study of Deep Learning infrastructure. RDL learns better whilst reducing human work needed by more than an order of magnitude.
• Score: 78.52438155603781
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present RelBench, a public benchmark for solving predictive tasks over relational databases with graph neural networks. RelBench provides databases and tasks spanning diverse domains and scales, and is intended to be a foundational infrastructure for future research. We use RelBench to conduct the first comprehensive study of Relational Deep Learning (RDL) (Fey et al., 2024), which combines graph neural network predictive models with (deep) tabular models that extract initial entity-level representations from raw tables. End-to-end learned RDL models fully exploit the predictive signal encoded in primary-foreign key links, marking a significant shift away from the dominant paradigm of manual feature engineering combined with tabular models. To thoroughly evaluate RDL against this prior gold-standard, we conduct an in-depth user study where an experienced data scientist manually engineers features for each task. In this study, RDL learns better models whilst reducing human work needed by more than an order of magnitude. This demonstrates the power of deep learning for solving predictive tasks over relational databases, opening up many new research opportunities enabled by RelBench.

Related papers

• A Pre-training Framework for Relational Data with Information-theoretic Principles [57.93973948947743]
  We introduce Task Vector Estimation (TVE), a novel pre-training framework that constructs supervisory signals via set-based aggregation over relational graphs.<n>TVE consistently outperforms traditional pre-training baselines.<n>Our findings advocate for pre-training objectives that encode task heterogeneity and temporal structure as design principles for predictive modeling on relational databases.
  arXiv  Detail & Related papers  (2025-07-14T00:17:21Z)
• Task-Agnostic Contrastive Pretraining for Relational Deep Learning [0.0]
  We propose a novel task-agnostic contrastive pretraining approach for RDL that enables database-wide representation learning.<n>We implement the respective pretraining approach through a modular RDL architecture.<n>Our preliminary results demonstrate that finetuning the pretrained models measurably outperforms training from scratch.
  arXiv  Detail & Related papers  (2025-06-27T13:18:13Z)
• REDELEX: A Framework for Relational Deep Learning Exploration [0.0]
  Recently, Deep Deep Learning has emerged as a novel paradigm wherein RDBs are conceptualized as graph structures.<n>There is a lack of analysis into the relationships between various RDL models and the characteristics of the underlying RDBs.<n>We present REDELEX$-$a comprehensive exploration framework for evaluating RDL models of varying complexity on the most diverse collection of over 70 RDBs.
  arXiv  Detail & Related papers  (2025-06-27T13:05:15Z)
• Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures [50.46688111973999]
  Graph machine learning has led to a significant increase in the capabilities of models that learn on arbitrary graph-structured data.<n>We present a new blueprint that enables end-to-end representation of'relational entity graphs' without traditional engineering feature.<n>We discuss key challenges including large-scale multi-table integration and the complexities of modeling temporal dynamics and heterogeneous data.
  arXiv  Detail & Related papers  (2025-06-19T23:51:38Z)
• Representation Learning for Tabular Data: A Comprehensive Survey [23.606506938919605]
  Tabular data, structured as rows and columns, is among the most prevalent data types in machine learning classification and regression applications. Deep Neural Networks (DNNs) have recently demonstrated promising results through their capability of representation learning. We organize existing methods into three main categories according to their generalization capabilities.
  arXiv  Detail & Related papers  (2025-04-17T17:58:23Z)
• Meta-Statistical Learning: Supervised Learning of Statistical Inference [59.463430294611626]
  This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks. We propose meta-statistical learning, a framework inspired by multi-instance learning that reformulates statistical inference tasks as supervised learning problems.
  arXiv  Detail & Related papers  (2025-02-17T18:04:39Z)
• RelGNN: Composite Message Passing for Relational Deep Learning [56.48834369525997]
  We introduce RelGNN, a novel GNN framework specifically designed to leverage the unique structural characteristics of the graphs built from relational databases.<n>RelGNN is evaluated on 30 diverse real-world tasks from Relbench (Fey et al., 2024), and achieves state-of-the-art performance on the vast majority tasks, with improvements of up to 25%.
  arXiv  Detail & Related papers  (2025-02-10T18:58:40Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)
• Relational Deep Learning: Graph Representation Learning on Relational Databases [69.7008152388055]
  We introduce an end-to-end representation approach to learn on data laid out across multiple tables. Message Passing Graph Neural Networks can then automatically learn across the graph to extract representations that leverage all data input.
  arXiv  Detail & Related papers  (2023-12-07T18:51:41Z)
• GFS: Graph-based Feature Synthesis for Prediction over Relational Databases [39.975491511390985]
  We propose a novel framework called Graph-based Feature Synthesis (GFS) GFS formulates relational database as a heterogeneous graph database. In an experiment over four real-world multi-table relational databases, GFS outperforms previous methods designed for relational databases.
  arXiv  Detail & Related papers  (2023-12-04T16:54:40Z)
• TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023 [33.70333110327871]
  We present TabR -- essentially, a feed-forward network with a custom k-Nearest-Neighbors-like component in the middle. On a set of public benchmarks with datasets up to several million objects, TabR demonstrates the best average performance. In addition to the much higher performance, TabR is simple and significantly more efficient.
  arXiv  Detail & Related papers  (2023-07-26T17:58:07Z)
• Relational Extraction on Wikipedia Tables using Convolutional and Memory Networks [6.200672130699805]
  Relation extraction (RE) is the task of extracting relations between entities in text. We introduce a new model consisting of Convolutional Neural Network (CNN) and Bidirectional-Long Short Term Memory (BiLSTM) network to encode entities.
  arXiv  Detail & Related papers  (2023-07-11T22:36:47Z)
• Learning from Context or Names? An Empirical Study on Neural Relation Extraction [112.06614505580501]
  We study the effect of two main information sources in text: textual context and entity mentions (names) We propose an entity-masked contrastive pre-training framework for relation extraction (RE) Our framework can improve the effectiveness and robustness of neural models in different RE scenarios.
  arXiv  Detail & Related papers  (2020-10-05T11:21:59Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier [58.979104709647295]
  We bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2019-12-27T02:05:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.