A Benchmarking Study of Kolmogorov-Arnold Networks on Tabular Data

• URL: http://arxiv.org/abs/2406.14529v1
• Date: Thu, 20 Jun 2024 17:41:34 GMT
• Title: A Benchmarking Study of Kolmogorov-Arnold Networks on Tabular Data
• Abstract summary: Kolmogorov-Arnold Networks (KANs) have been introduced into the world of machine learning, quickly capturing the attention of the entire community. We present a benchmarking study comparing KANs and Multi-Layer Perceptrons (MLPs) on datasets. KANs demonstrate superior or comparable accuracy and F1 scores, excelling particularly in datasets with numerous instances.
• Score: 14.654403764632594
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kolmogorov-Arnold Networks (KANs) have very recently been introduced into the world of machine learning, quickly capturing the attention of the entire community. However, KANs have mostly been tested for approximating complex functions or processing synthetic data, while a test on real-world tabular datasets is currently lacking. In this paper, we present a benchmarking study comparing KANs and Multi-Layer Perceptrons (MLPs) on tabular datasets. The study evaluates task performance and training times. From the results obtained on the various datasets, KANs demonstrate superior or comparable accuracy and F1 scores, excelling particularly in datasets with numerous instances, suggesting robust handling of complex data. We also highlight that this performance improvement of KANs comes with a higher computational cost when compared to MLPs of comparable sizes.

Related papers

• Is the Statistical Advantage Worth the Cost? An Empirical Comparison of KANs and MLPs for Structured Data Classification [0.0]
  This study presents an empirical benchmarking comparison between Kolmogorov-Arnold Networks (KANs) and Multi-Layer Perceptrons (MLPs)<n>We evaluate their out-of-the-box performance on twelve publicly available datasets spanning binary, multiclass, multilabel, and ordinal problems.
  arXiv  Detail & Related papers  (2026-07-15T03:30:46Z)
• CLIMB: Class-imbalanced Learning Benchmark on Tabular Data [68.07599497425267]
  Class-imbalanced learning (CIL) is important in many real-world applications where the minority class holds the critical but rare outcomes.<n>In this paper, we present CLIMB, a comprehensive benchmark for class-imbalanced learning.<n> CLIMB includes 73 real-world datasets across diverse domains and imbalance levels, along with unified implementations of 29 representative CIL algorithms.
  arXiv  Detail & Related papers  (2025-05-23T04:21:03Z)
• Enhancing Federated Learning with Kolmogorov-Arnold Networks: A Comparative Study Across Diverse Aggregation Strategies [0.24578723416255752]
  Kolmogorov-Arnold Networks (KAN) have shown promising capabilities in modeling complex nonlinear relationships.<n>KANs consistently outperform Multilayer Perceptrons in terms of accuracy, stability, and convergence efficiency.
  arXiv  Detail & Related papers  (2025-05-12T14:56:27Z)
• Kolmogorov-Arnold Network Autoencoders [0.0]
  Kolmogorov-Arnold Networks (KANs) are promising alternatives to Multi-Layer Perceptrons (MLPs) KANs align closely with the Kolmogorov-Arnold representation theorem, potentially enhancing both model accuracy and interpretability. Our results demonstrate that KAN-based autoencoders achieve competitive performance in terms of reconstruction accuracy.
  arXiv  Detail & Related papers  (2024-10-02T22:56:00Z)
• A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networks [43.70716358136333]
  Kolmogorov- Networks (KAN) are based on a fundamentally different mathematical framework. KANs address several major issues insio, such as forgetting in continual learning scenarios. We extend the investigation by evaluating the performance of KANs in continual learning tasks within computer vision.
  arXiv  Detail & Related papers  (2024-09-20T14:49:21Z)
• TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks [30.922069185335246]
  We find two common characteristics of tabular data in typical industrial applications that are underrepresented in the datasets usually used for evaluation in the literature. A considerable portion of datasets in production settings stem from extensive data acquisition and feature engineering pipelines. This can have an impact on the absolute and relative number of predictive, uninformative, and correlated features compared to academic datasets.
  arXiv  Detail & Related papers  (2024-06-27T17:55:31Z)
• SMaRt: Improving GANs with Score Matching Regularity [94.81046452865583]
  Generative adversarial networks (GANs) usually struggle in learning from highly diverse data, whose underlying manifold is complex. We show that score matching serves as a promising solution to this issue thanks to its capability of persistently pushing the generated data points towards the real data manifold. We propose to improve the optimization of GANs with score matching regularity (SMaRt)
  arXiv  Detail & Related papers  (2023-11-30T03:05:14Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Class Balancing GAN with a Classifier in the Loop [58.29090045399214]
  We introduce a novel theoretically motivated Class Balancing regularizer for training GANs. Our regularizer makes use of the knowledge from a pre-trained classifier to ensure balanced learning of all the classes in the dataset. We demonstrate the utility of our regularizer in learning representations for long-tailed distributions via achieving better performance than existing approaches over multiple datasets.
  arXiv  Detail & Related papers  (2021-06-17T11:41:30Z)
• Comparing Test Sets with Item Response Theory [53.755064720563]
  We evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples. We find that Quoref, HellaSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models. We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
  arXiv  Detail & Related papers  (2021-06-01T22:33:53Z)
• Benchmarking the Benchmark -- Analysis of Synthetic NIDS Datasets [4.125187280299247]
  We analyse the statistical properties of benign traffic in three of the more recent and relevant NIDS datasets. Our results show a distinct difference of most of the considered statistical features between the synthetic datasets and two real-world datasets.
  arXiv  Detail & Related papers  (2021-04-19T03:17:37Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• RadixSpline: A Single-Pass Learned Index [84.84747738666263]
  We introduce RadixSpline (RS), a learned index that can be built in a single pass over the data. RS achieves competitive results on all datasets, despite the fact that it only has two parameters.
  arXiv  Detail & Related papers  (2020-04-30T01:56:54Z)
• A Close Look at Deep Learning with Small Data [0.0]
  We show that model complexity is a critical factor when only a few samples per class are available. We also show that even standard data augmentation can boost recognition performance by large margins.
  arXiv  Detail & Related papers  (2020-03-28T17:11:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.