Beyond Tree Models: A Hybrid Model of KAN and gMLP for Large-Scale Financial Tabular Data

• URL: http://arxiv.org/abs/2412.02097v1
• Date: Tue, 03 Dec 2024 02:38:07 GMT
• Title: Beyond Tree Models: A Hybrid Model of KAN and gMLP for Large-Scale Financial Tabular Data
• Authors: Mingming Zhang, Jiahao Hu, Pengfei Shi, Ningtao Wang, Ruizhe Gao, Guandong Sun, Feng Zhao, Yulin kang, Xing Fu, Weiqiang Wang, Junbo Zhao,
• Abstract summary: TKGMLP is a hybrid network for tabular data that combines shallow Kolmogorov Arnold Networks with Gated Multilayer Perceptron. We validate TKGMLP on a real-world credit scoring dataset, where it achieves state-of-the-art results and outperforms current benchmarks. We propose a novel feature encoding method for numerical data, specifically designed to address the predominance of numerical features in financial datasets.
• Score: 28.34587057844627
• License:
• Abstract: Tabular data plays a critical role in real-world financial scenarios. Traditionally, tree models have dominated in handling tabular data. However, financial datasets in the industry often encounter some challenges, such as data heterogeneity, the predominance of numerical features and the large scale of the data, which can range from tens of millions to hundreds of millions of records. These challenges can lead to significant memory and computational issues when using tree-based models. Consequently, there is a growing need for neural network-based solutions that can outperform these models. In this paper, we introduce TKGMLP, an hybrid network for tabular data that combines shallow Kolmogorov Arnold Networks with Gated Multilayer Perceptron. This model leverages the strengths of both architectures to improve performance and scalability. We validate TKGMLP on a real-world credit scoring dataset, where it achieves state-of-the-art results and outperforms current benchmarks. Furthermore, our findings demonstrate that the model continues to improve as the dataset size increases, making it highly scalable. Additionally, we propose a novel feature encoding method for numerical data, specifically designed to address the predominance of numerical features in financial datasets. The integration of this feature encoding method within TKGMLP significantly improves prediction accuracy. This research not only advances table prediction technology but also offers a practical and effective solution for handling large-scale numerical tabular data in various industrial applications.

Related papers

• Class-Imbalanced-Aware Adaptive Dataset Distillation for Scalable Pretrained Model on Credit Scoring [10.737033782376905]
  We present a novel framework for scaling up the application of large pretrained models on financial datasets. We integrate the imbalance-aware techniques during dataset distillation, resulting in improved performance in financial datasets.
  arXiv  Detail & Related papers  (2025-01-18T06:59:36Z)
• GNN Applied to Ego-nets for Friend Suggestions [0.0]
  We introduce the Generalized-network Friendship Score framework, which makes it possible to use complex supervised models without sacrificing scalability. The underlying model takes an ego-net as input and produces a pairwise relevance matrix for its nodes. In addition, we develop the WalkGNN model which is capable of working effectively in the social network domain.
  arXiv  Detail & Related papers  (2024-12-16T15:37:17Z)
• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• Escaping the Forest: Sparse Interpretable Neural Networks for Tabular Data [0.0]
  We show that our models, Sparse TABular NET or sTAB-Net with attention mechanisms, are more effective than tree-based models. They achieve better performance than post-hoc methods like SHAP.
  arXiv  Detail & Related papers  (2024-10-23T10:50:07Z)
• A Survey on Deep Tabular Learning [0.0]
  Tabular data presents unique challenges for deep learning due to its heterogeneous nature and lack of spatial structure. This survey reviews the evolution of deep learning models for Tabular data, from early fully connected networks (FCNs) to advanced architectures like TabNet, SAINT, TabTranSELU, and MambaNet.
  arXiv  Detail & Related papers  (2024-10-15T20:08:08Z)
• Scaling Laws Do Not Scale [54.72120385955072]
  Recent work has argued that as the size of a dataset increases, the performance of a model trained on that dataset will increase. We argue that this scaling law relationship depends on metrics used to measure performance that may not correspond with how different groups of people perceive the quality of models' output. Different communities may also have values in tension with each other, leading to difficult, potentially irreconcilable choices about metrics used for model evaluations.
  arXiv  Detail & Related papers  (2023-07-05T15:32:21Z)
• A Solvable Model of Neural Scaling Laws [72.8349503901712]
  Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws. We propose a statistical model -- a joint generative data model and random feature model -- that captures this neural scaling phenomenology. Key findings are the manner in which the power laws that occur in the statistics of natural datasets are extended by nonlinear random feature maps.
  arXiv  Detail & Related papers  (2022-10-30T15:13:18Z)
• Graph-Regularized Tensor Regression: A Domain-Aware Framework for Interpretable Multi-Way Financial Modelling [23.030263841031633]
  We develop a novel Graph-Regularized Regression (GRTR) framework, whereby knowledge about cross-asset relations is incorporated into the model in the form of a graph Laplacian matrix. By virtue of tensor algebra, the proposed framework is shown to be fully interpretable, both coefficient-wise and dimension-wise. The GRTR model is validated in a multi-way financial forecasting setting and is shown to achieve improved performance at reduced computational costs.
  arXiv  Detail & Related papers  (2022-10-26T13:39:08Z)
• Improving Classifier Training Efficiency for Automatic Cyberbullying Detection with Feature Density [58.64907136562178]
  We study the effectiveness of Feature Density (FD) using different linguistically-backed feature preprocessing methods. We hypothesise that estimating dataset complexity allows for the reduction of the number of required experiments. The difference in linguistic complexity of datasets allows us to additionally discuss the efficacy of linguistically-backed word preprocessing.
  arXiv  Detail & Related papers  (2021-11-02T15:48:28Z)
• CHEER: Rich Model Helps Poor Model via Knowledge Infusion [69.23072792708263]
  We develop a knowledge infusion framework named CHEER that can succinctly summarize such rich model into transferable representations. Our empirical results showed that CHEER outperformed baselines by 5.60% to 46.80% in terms of the macro-F1 score on multiple physiological datasets.
  arXiv  Detail & Related papers  (2020-05-21T21:44:21Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.