SPARE: A Single-Pass Neural Model for Relational Databases
- URL: http://arxiv.org/abs/2310.13581v1
- Date: Fri, 20 Oct 2023 15:23:17 GMT
- Title: SPARE: A Single-Pass Neural Model for Relational Databases
- Authors: Benjamin Hilprecht, Kristian Kersting and Carsten Binnig
- Abstract summary: We propose SPARE, a new class of neural models that can be trained efficiently on RDBs while providing similar accuracies as GNNs.
For enabling efficient training, different from GNNs, SPARE makes use of the fact that data in RDBs has a predictive regular structure, which allows one to train these models in a single pass while exploiting symmetries at the same time.
- Score: 36.55513135391452
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While there has been extensive work on deep neural networks for images and
text, deep learning for relational databases (RDBs) is still a rather
unexplored field.
One direction that recently gained traction is to apply Graph Neural Networks
(GNNs) to RBDs. However, training GNNs on large relational databases (i.e.,
data stored in multiple database tables) is rather inefficient due to multiple
rounds of training and potentially large and inefficient representations.
Hence, in this paper we propose SPARE (Single-Pass Relational models), a new
class of neural models that can be trained efficiently on RDBs while providing
similar accuracies as GNNs. For enabling efficient training, different from
GNNs, SPARE makes use of the fact that data in RDBs has a regular structure,
which allows one to train these models in a single pass while exploiting
symmetries at the same time. Our extensive empirical evaluation demonstrates
that SPARE can significantly speedup both training and inference while offering
competitive predictive performance over numerous baselines.
Related papers
- Team up GBDTs and DNNs: Advancing Efficient and Effective Tabular Prediction with Tree-hybrid MLPs [20.67800392863432]
Tabular datasets play a crucial role in various applications.
Two prominent model types, Boosted Decision Trees (GBDTs) and Deep Neural Networks (DNNs), have demonstrated performance advantages on distinct prediction tasks.
This paper proposes a new framework that amalgamates the advantages of both GBDTs and DNNs, resulting in a DNN algorithm that is as efficient as GBDTs and is competitively effective regardless of dataset preferences.
arXiv Detail & Related papers (2024-07-13T07:13:32Z) - Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of One [60.5818387068983]
Graph neural networks (GNN) suffer from severe inefficiency.
We propose to decouple a multi-layer GNN as multiple simple modules for more efficient training.
We show that the proposed framework is highly efficient with reasonable performance.
arXiv Detail & Related papers (2023-04-20T07:21:32Z) - Poster: Link between Bias, Node Sensitivity and Long-Tail Distribution
in trained DNNs [12.404169549562523]
Training datasets with long-tail distribution pose a challenge for deep neural networks (DNNs)
This work identifies the node bias that leads to a varying sensitivity of the nodes for different output classes.
We support our reasoning using an empirical case study of the networks trained on a real-world dataset.
arXiv Detail & Related papers (2023-03-29T10:49:31Z) - Learn, Unlearn and Relearn: An Online Learning Paradigm for Deep Neural
Networks [12.525959293825318]
We introduce Learn, Unlearn, and Relearn (LURE) an online learning paradigm for deep neural networks (DNNs)
LURE interchanges between the unlearning phase, which selectively forgets the undesirable information in the model, and the relearning phase, which emphasizes learning on generalizable features.
We show that our training paradigm provides consistent performance gains across datasets in both classification and few-shot settings.
arXiv Detail & Related papers (2023-03-18T16:45:54Z) - Online Evolutionary Neural Architecture Search for Multivariate
Non-Stationary Time Series Forecasting [72.89994745876086]
This work presents the Online Neuro-Evolution-based Neural Architecture Search (ONE-NAS) algorithm.
ONE-NAS is a novel neural architecture search method capable of automatically designing and dynamically training recurrent neural networks (RNNs) for online forecasting tasks.
Results demonstrate that ONE-NAS outperforms traditional statistical time series forecasting methods.
arXiv Detail & Related papers (2023-02-20T22:25:47Z) - Recurrent Bilinear Optimization for Binary Neural Networks [58.972212365275595]
BNNs neglect the intrinsic bilinear relationship of real-valued weights and scale factors.
Our work is the first attempt to optimize BNNs from the bilinear perspective.
We obtain robust RBONNs, which show impressive performance over state-of-the-art BNNs on various models and datasets.
arXiv Detail & Related papers (2022-09-04T06:45:33Z) - Simple Recurrent Neural Networks is all we need for clinical events
predictions using EHR data [22.81278657120305]
Recurrent neural networks (RNNs) are common architecture for EHR-based clinical events predictive models.
We used two prediction tasks: the risk for developing heart failure and the risk of early readmission for inpatient hospitalization.
We found that simple gated RNN models, including GRUs and LSTMs, often offer competitive results when properly tuned with Bayesian Optimization.
arXiv Detail & Related papers (2021-10-03T13:07:23Z) - Shift-Robust GNNs: Overcoming the Limitations of Localized Graph
Training data [52.771780951404565]
Shift-Robust GNN (SR-GNN) is designed to account for distributional differences between biased training data and the graph's true inference distribution.
We show that SR-GNN outperforms other GNN baselines by accuracy, eliminating at least (40%) of the negative effects introduced by biased training data.
arXiv Detail & Related papers (2021-08-02T18:00:38Z) - Deep Time Delay Neural Network for Speech Enhancement with Full Data
Learning [60.20150317299749]
This paper proposes a deep time delay neural network (TDNN) for speech enhancement with full data learning.
To make full use of the training data, we propose a full data learning method for speech enhancement.
arXiv Detail & Related papers (2020-11-11T06:32:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.