Exploring the Potential of Polynomial Basis Functions in Kolmogorov-Arnold Networks: A Comparative Study of Different Groups of Polynomials

• URL: http://arxiv.org/abs/2406.02583v1
• Date: Thu, 30 May 2024 20:40:16 GMT
• Title: Exploring the Potential of Polynomial Basis Functions in Kolmogorov-Arnold Networks: A Comparative Study of Different Groups of Polynomials
• Authors: Seyd Teymoor Seydi,
• Abstract summary: This paper presents a survey of 18 distincts and their potential applications in Gottliebmogorov Network (KAN) models. The study aims to investigate the suitability of theseDistincts as basis functions in KAN models for complex tasks like handwritten digit classification. The performance metrics of the KAN models, including overall accuracy, Kappa, and F1 score, are evaluated and compared.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a comprehensive survey of 18 distinct polynomials and their potential applications in Kolmogorov-Arnold Network (KAN) models as an alternative to traditional spline-based methods. The polynomials are classified into various groups based on their mathematical properties, such as orthogonal polynomials, hypergeometric polynomials, q-polynomials, Fibonacci-related polynomials, combinatorial polynomials, and number-theoretic polynomials. The study aims to investigate the suitability of these polynomials as basis functions in KAN models for complex tasks like handwritten digit classification on the MNIST dataset. The performance metrics of the KAN models, including overall accuracy, Kappa, and F1 score, are evaluated and compared. The Gottlieb-KAN model achieves the highest performance across all metrics, suggesting its potential as a suitable choice for the given task. However, further analysis and tuning of these polynomials on more complex datasets are necessary to fully understand their capabilities in KAN models. The source code for the implementation of these KAN models is available at https://github.com/seydi1370/Basis_Functions .

Related papers

• GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations [1.1142444517901018]
  We propose GINN-LP, an interpretable neural network, to discover the form of a Laurent Polynomial equation. To the best of our knowledge, this is the first neural network that can discover arbitrary terms without any prior information on the order. We show that GINN-LP outperforms the state-of-theart symbolic regression methods on datasets.
  arXiv  Detail & Related papers  (2023-12-18T03:44:29Z)
• Learning and Generalizing Polynomials in Simulation Metamodeling [7.41244589428771]
  This paper collects and proposes multiplicative (MNN) architectures for approxing higher-order neural networks. Experiments show that MNNs are better than baseline models at generalizing, and their performance in validation is true to their performance in out-of-distribution tests.
  arXiv  Detail & Related papers  (2023-07-20T14:11:29Z)
• Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks [40.137069931650444]
  Tropical geometry has recently found several applications in the analysis of neural networks with piecewise linear activation functions. This paper presents a new look at the problem of tropical division and its application to the simplification of neural networks.
  arXiv  Detail & Related papers  (2023-06-27T02:26:07Z)
• Graph Neural Networks with Learnable and Optimal Polynomial Bases [14.8308791628821]
  Polynomial filters, a kind of Graph Networks, typically use a predetermined basis and learn the coefficients from the training data. Can we learn a suitable basis from the training data? Can we determine the optimal basis for a given graph and a node?
  arXiv  Detail & Related papers  (2023-02-24T03:24:04Z)
• PAC Reinforcement Learning for Predictive State Representations [60.00237613646686]
  We study online Reinforcement Learning (RL) in partially observable dynamical systems. We focus on the Predictive State Representations (PSRs) model, which is an expressive model that captures other well-known models. We develop a novel model-based algorithm for PSRs that can learn a near optimal policy in sample complexity scalingly.
  arXiv  Detail & Related papers  (2022-07-12T17:57:17Z)
• Partial Counterfactual Identification from Observational and Experimental Data [83.798237968683]
  We develop effective Monte Carlo algorithms to approximate the optimal bounds from an arbitrary combination of observational and experimental data. Our algorithms are validated extensively on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-10-12T02:21:30Z)
• Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models [55.41644538483948]
  In modern data science, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. We use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties.
  arXiv  Detail & Related papers  (2021-07-07T11:17:09Z)
• Bilinear Classes: A Structural Framework for Provable Generalization in RL [119.42509700822484]
  Bilinear Classes is a new structural framework which permits generalization in reinforcement learning. The framework incorporates nearly all existing models in which a sample complexity is achievable. Our main result provides an RL algorithm which has sample complexity for Bilinear Classes.
  arXiv  Detail & Related papers  (2021-03-19T16:34:20Z)
• Finite-Function-Encoding Quantum States [52.77024349608834]
  We introduce finite-function-encoding (FFE) states which encode arbitrary $d$-valued logic functions. We investigate some of their structural properties.
  arXiv  Detail & Related papers  (2020-12-01T13:53:23Z)
• On Polynomial Approximations for Privacy-Preserving and Verifiable ReLU Networks [6.130998208629276]
  We propose a degree-2 activation function with a first order term and empirically show that it can lead to much better models. Our proposed function improves the test accuracy by up to 10.4% compared to the square function.
  arXiv  Detail & Related papers  (2020-11-11T03:32:22Z)
• From Sets to Multisets: Provable Variational Inference for Probabilistic Integer Submodular Models [82.95892656532696]
  Submodular functions have been studied extensively in machine learning and data mining. In this work, we propose a continuous DR-submodular extension for integer submodular functions. We formulate a new probabilistic model which is defined through integer submodular functions.
  arXiv  Detail & Related papers  (2020-06-01T22:20:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.