Smooth Kolmogorov Arnold networks enabling structural knowledge representation

• URL: http://arxiv.org/abs/2405.11318v2
• Date: Mon, 27 May 2024 09:32:35 GMT
• Title: Smooth Kolmogorov Arnold networks enabling structural knowledge representation
• Authors: Moein E. Samadi, Younes Müller, Andreas Schuppert,
• Abstract summary: Kolmogorov-Arnold Networks (KANs) offer an efficient and interpretable alternative to traditional multi-layer perceptron (MLP) architectures. By leveraging inherent structural knowledge, KANs may reduce the data required for training and mitigate the risk of generating hallucinated predictions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kolmogorov-Arnold Networks (KANs) offer an efficient and interpretable alternative to traditional multi-layer perceptron (MLP) architectures due to their finite network topology. However, according to the results of Kolmogorov and Vitushkin, the representation of generic smooth functions by KAN implementations using analytic functions constrained to a finite number of cutoff points cannot be exact. Hence, the convergence of KAN throughout the training process may be limited. This paper explores the relevance of smoothness in KANs, proposing that smooth, structurally informed KANs can achieve equivalence to MLPs in specific function classes. By leveraging inherent structural knowledge, KANs may reduce the data required for training and mitigate the risk of generating hallucinated predictions, thereby enhancing model reliability and performance in computational biomedicine.

Related papers

• Kolmogorov-Arnold Network Autoencoders [0.0]
  Kolmogorov-Arnold Networks (KANs) are promising alternatives to Multi-Layer Perceptrons (MLPs) KANs align closely with the Kolmogorov-Arnold representation theorem, potentially enhancing both model accuracy and interpretability. Our results demonstrate that KAN-based autoencoders achieve competitive performance in terms of reconstruction accuracy.
  arXiv  Detail & Related papers  (2024-10-02T22:56:00Z)
• Kolmogorov-Arnold Network for Online Reinforcement Learning [0.22615818641180724]
  Kolmogorov-Arnold Networks (KANs) have shown potential as an alternative to Multi-Layer Perceptrons (MLPs) in neural networks. KANs provide universal function approximation with fewer parameters and reduced memory usage.
  arXiv  Detail & Related papers  (2024-08-09T03:32:37Z)
• F-KANs: Federated Kolmogorov-Arnold Networks [3.8277268808551512]
  We present an innovative federated learning (FL) approach that utilizes Kolmogorov-Arnold Networks (KANs) for classification tasks. The study evaluates the performance of federated KANs compared to traditional Multi-Layer Perceptrons (MLPs) classification task.
  arXiv  Detail & Related papers  (2024-07-29T15:28:26Z)
• U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation [48.40120035775506]
  Kolmogorov-Arnold Networks (KANs) reshape the neural network learning via the stack of non-linear learnable activation functions. We investigate, modify and re-design the established U-Net pipeline by integrating the dedicated KAN layers on the tokenized intermediate representation, termed U-KAN. We further delved into the potential of U-KAN as an alternative U-Net noise predictor in diffusion models, demonstrating its applicability in generating task-oriented model architectures.
  arXiv  Detail & Related papers  (2024-06-05T04:13:03Z)
• Kolmogorov-Arnold Networks (KANs) for Time Series Analysis [6.932243286441558]
  We introduce a novel application of Kolmogorov-Arnold Networks (KANs) to time series forecasting. Inspired by the Kolmogorov-Arnold representation theorem, KANs replace traditional linear weights with spline-parametrized univariate functions. We demonstrate that KANs outperforms conventional Multi-Layer Perceptrons (MLPs) in a real-world satellite traffic forecasting task.
  arXiv  Detail & Related papers  (2024-05-14T17:38:17Z)
• Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation [0.0]
  This paper presents the Chebyshev Kolmogorov-Arnold Network (Chebyshev KAN), a new neural network architecture inspired by the Kolmogorov-Arnold theorem. By utilizing learnable functions parametrized by Chebyshevs on the network's edges, Chebyshev KANs enhance flexibility, efficiency, and interpretability in function approximation tasks.
  arXiv  Detail & Related papers  (2024-05-12T07:55:43Z)
• Layer-wise Feedback Propagation [53.00944147633484]
  We present Layer-wise Feedback Propagation (LFP), a novel training approach for neural-network-like predictors. LFP assigns rewards to individual connections based on their respective contributions to solving a given task. We demonstrate its effectiveness in achieving comparable performance to gradient descent on various models and datasets.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)
• Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks [82.03459331544737]
  We study the performance of ConvResNeXts, trained with weight decay from the perspective of nonparametric classification. Our analysis allows for infinitely many building blocks in ConvResNeXts, and shows that weight decay implicitly enforces sparsity on these blocks.
  arXiv  Detail & Related papers  (2023-07-04T11:08:03Z)
• Semantic Probabilistic Layers for Neuro-Symbolic Learning [83.25785999205932]
  We design a predictive layer for structured-output prediction (SOP) It can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints. Our Semantic Probabilistic Layer (SPL) can model intricate correlations, and hard constraints, over a structured output space.
  arXiv  Detail & Related papers  (2022-06-01T12:02:38Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Local Propagation in Constraint-based Neural Network [77.37829055999238]
  We study a constraint-based representation of neural network architectures. We investigate a simple optimization procedure that is well suited to fulfil the so-called architectural constraints.
  arXiv  Detail & Related papers  (2020-02-18T16:47:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.