A Fair Benchmarking of Deep Relational Database Learning Models

• URL: http://arxiv.org/abs/2607.03659v1
• Date: Sat, 04 Jul 2026 01:20:21 GMT
• Title: A Fair Benchmarking of Deep Relational Database Learning Models
• Abstract summary: relational databases (RDBs) are the primary infrastructure in many enterprises.<n>Recent deep learning methods designed for RDBs have been evaluated under inconsistent experimental protocols.<n>We present one of the first systematic benchmarking studies of recently released deep learning methods for RDBs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Relational databases (RDBs) are the primary data infrastructure in many enterprises, yet recent deep learning methods designed for RDBs have been evaluated under inconsistent experimental protocols, making fair comparison difficult. We present one of the first systematic benchmarking studies of recently released deep learning methods for RDBs, evaluating them across five relational databases, with one classification and one regression task for each. We refactor all deep RDB models to allow the full range of experimental procedures to be applied consistently across all methods. Our findings indicate that the relational transformer (RT) approach delivers the strongest overall performance on both classification and regression tasks compared to the state-of-the-art graph-based modeling and learning of RDBs. Even for single-table learning tasks, deep learning methods designed for RDBs outperform the leading tabular foundation model, TabPFN 2.5. Extending learning from a single table (hop = 0) to multiple tables (hop = 1, 2) by connecting neighboring tables in relational databases enhances performance, but the additional benefit from higher hops diminishes as computational overhead grows. Deep RDB learning methods have the potential to challenge state-of-the-art tabular foundation models, especially on large-scale enterprise data. The source code for this benchmarking study is publicly available.

Related papers

• Advancing Open and Reproducible Relational Learning: RelArena-$α$, TabPFN-Rel and RPI [93.1293239420285]
  We open-source three pieces of software that we expect to accelerate research in the field towards meaningful real-world impact.<n>RelArena-$$ provides a unified framework for running and comparing baselines on RelBench v1.<n>We release the initial version of TabPFN-Rel, a purpose-built relational harness for TabPFN-3.
  arXiv  Detail & Related papers  (2026-08-17T09:25:05Z)
• RelPrism: A Multi-Faceted Pre-training Framework with Self-Generated Tasks for Relational Databases [49.47033473074821]
  We propose RelPrism, a multi-faceted self-supervised learning framework for RDBs.<n>RDB tasks often require multi-faceted information across different perspectives and granularities.<n>Our experiments show that RelPrism improves ROC-AUC by 4.15% for classification and reduces MAE by 10.75% for regression over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-05-22T05:19:58Z)
• Relational In-Context Learning via Synthetic Pre-training with Structural Prior [60.404256960057545]
  RDB-PFN is the first relational foundation model trained purely via $textbfsynthetic$.<n>Inspired by Prior-Data Fitted Networks (PFNs) where synthetic data generated from Structural Causal Models (SCMs) enables reasoning on single tables.<n>Experiments verify RDB-PFN achieves strong few-shot performance on 19 real-world prediction tasks.
  arXiv  Detail & Related papers  (2026-03-04T07:30:54Z)
• Relational Database Distillation: From Structured Tables to Condensed Graph Data [48.347717300340435]
  We aim to distill large-scale RDBs into compact heterogeneous graphs while retaining the power required for graph-based models.<n>We further design a kernel ridge regression-guided objective with pseudo-labels, which produces quality features for the distilled graph.
  arXiv  Detail & Related papers  (2025-10-08T13:05:31Z)
• Synthesize, Retrieve, and Propagate: A Unified Predictive Modeling Framework for Relational Databases [34.57267286892218]
  We propose SRP, a unified predictive modeling framework that synthesizes features using the unary dependency.<n>SRP is designed to fully capture both the unary and the composite dependencies within a relational database.
  arXiv  Detail & Related papers  (2025-08-10T07:59:41Z)
• REDELEX: A Framework for Relational Deep Learning Exploration [0.0]
  Recently, Deep Deep Learning has emerged as a novel paradigm wherein RDBs are conceptualized as graph structures.<n>There is a lack of analysis into the relationships between various RDL models and the characteristics of the underlying RDBs.<n>We present REDELEX$-$a comprehensive exploration framework for evaluating RDL models of varying complexity on the most diverse collection of over 70 RDBs.
  arXiv  Detail & Related papers  (2025-06-27T13:05:15Z)
• RDB2G-Bench: A Comprehensive Benchmark for Automatic Graph Modeling of Relational Databases [34.357399264742526]
  We introduce RDB2G-Bench, the first benchmark framework for evaluating graph modeling methods.<n>We benchmark 10 automatic RDB-to-graph modeling methods on 12 tasks about 380x faster than on-the-fly evaluation.<n>Our analysis of the datasets and benchmark results reveals key structural patterns affecting graph model effectiveness.
  arXiv  Detail & Related papers  (2025-06-02T06:34:10Z)
• RelBench: A Benchmark for Deep Learning on Relational Databases [78.52438155603781]
  We present RelBench, a public benchmark for solving tasks over databases with graph neural networks. We use RelBench to conduct the first comprehensive study of Deep Learning infrastructure. RDL learns better whilst reducing human work needed by more than an order of magnitude.
  arXiv  Detail & Related papers  (2024-07-29T14:46:13Z)
• 4DBInfer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on Relational DBs [67.47600679176963]
  RDBs store vast amounts of rich, informative data spread across interconnected tables. The progress of predictive machine learning models falls behind advances in other domains such as computer vision or natural language processing. We explore a class of baseline models predicated on converting multi-table datasets into graphs. We assemble a diverse collection of large-scale RDB datasets and (ii) coincident predictive tasks.
  arXiv  Detail & Related papers  (2024-04-28T15:04:54Z)
• RDBench: ML Benchmark for Relational Databases [38.96429447951739]
  We introduce RDBench, a standardized benchmark to promote reproducible machine learning (ML) research on relational databases (RDBs) RDBench offers diverse RDB datasets of varying scales, domains, graphs and relational structures, organized into 4 levels. For the first time, RDBench enables meaningful comparisons between ML methods from diverse domains, ranging from XBoostG to Graph Neural Networks, under RDB prediction tasks.
  arXiv  Detail & Related papers  (2023-10-25T17:59:34Z)
• Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study [51.33182775762785]
  This paper presents an empirical study to build relation extraction systems in low-resource settings. We investigate three schemes to evaluate the performance in low-resource settings: (i) different types of prompt-based methods with few-shot labeled data; (ii) diverse balancing methods to address the long-tailed distribution issue; and (iii) data augmentation technologies and self-training to generate more labeled in-domain data.
  arXiv  Detail & Related papers  (2022-10-19T15:46:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.