Kolmogorov-Arnold Networks (KAN) for Time Series Classification and Robust Analysis

• URL: http://arxiv.org/abs/2408.07314v3
• Date: Wed, 11 Sep 2024 05:10:12 GMT
• Title: Kolmogorov-Arnold Networks (KAN) for Time Series Classification and Robust Analysis
• Authors: Chang Dong, Liangwei Zheng, Weitong Chen,
• Abstract summary: Kolmogorov-Arnold Networks (KAN) has attracted significant attention as a promising alternative to traditional Multi-Layer Perceptrons (MLP) Despite their theoretical appeal, KAN require validation on large-scale benchmark datasets.
• Score: 2.978024452652925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Kolmogorov-Arnold Networks (KAN) has recently attracted significant attention as a promising alternative to traditional Multi-Layer Perceptrons (MLP). Despite their theoretical appeal, KAN require validation on large-scale benchmark datasets. Time series data, which has become increasingly prevalent in recent years, especially univariate time series are naturally suited for validating KAN. Therefore, we conducted a fair comparison among KAN, MLP, and mixed structures. The results indicate that KAN can achieve performance comparable to, or even slightly better than, MLP across 128 time series datasets. We also performed an ablation study on KAN, revealing that the output is primarily determined by the base component instead of b-spline function. Furthermore, we assessed the robustness of these models and found that KAN and the hybrid structure MLP\_KAN exhibit significant robustness advantages, attributed to their lower Lipschitz constants. This suggests that KAN and KAN layers hold strong potential to be robust models or to improve the adversarial robustness of other models.

Related papers

• On the expressiveness and spectral bias of KANs [17.42614039265962]
  KANs were recently proposed as a potential alternative to the prevalent architectural backbone of many deep learning models, the multi-layer perceptron (MLP) KANs have seen success in various tasks of AI for science, with their empirical efficiency and accuracy demostrated in function regression, PDE solving, and many more scientific problems.
  arXiv  Detail & Related papers  (2024-10-02T17:57:38Z)
• A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networks [43.70716358136333]
  Kolmogorov- Networks (KAN) are based on a fundamentally different mathematical framework. KANs address several major issues insio, such as forgetting in continual learning scenarios. We extend the investigation by evaluating the performance of KANs in continual learning tasks within computer vision.
  arXiv  Detail & Related papers  (2024-09-20T14:49:21Z)
• KAN4TSF: Are KAN and KAN-based models Effective for Time Series Forecasting? [20.483074918879133]
  We introduce the Kolmogorov-Arnold Network (KAN) into time series forecasting research. We propose the Reversible Mixture of KAN experts (RMoK) model, which is a KAN-based model for time series forecasting. We find the relationship between temporal feature weights and data periodicity through visualization.
  arXiv  Detail & Related papers  (2024-08-21T03:21:52Z)
• F-KANs: Federated Kolmogorov-Arnold Networks [3.8277268808551512]
  We present an innovative federated learning (FL) approach that utilizes Kolmogorov-Arnold Networks (KANs) for classification tasks. The study evaluates the performance of federated KANs compared to traditional Multi-Layer Perceptrons (MLPs) classification task.
  arXiv  Detail & Related papers  (2024-07-29T15:28:26Z)
• KAN or MLP: A Fairer Comparison [63.794304207664176]
  This paper offers a fairer and more comprehensive comparison of KAN and models across various tasks. We control the number of parameters and FLOPs to compare the performance of KAN and representation. We find that KAN's issue is more severe than that of forgetting in a standard class-incremental continual learning setting.
  arXiv  Detail & Related papers  (2024-07-23T17:43:35Z)
• Smooth Kolmogorov Arnold networks enabling structural knowledge representation [0.0]
  Kolmogorov-Arnold Networks (KANs) offer an efficient and interpretable alternative to traditional multi-layer perceptron (MLP) architectures. By leveraging inherent structural knowledge, KANs may reduce the data required for training and mitigate the risk of generating hallucinated predictions.
  arXiv  Detail & Related papers  (2024-05-18T15:27:14Z)
• Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs [50.25683648762602]
  We introduce Koopman VAE, a new generative framework that is based on a novel design for the model prior. Inspired by Koopman theory, we represent the latent conditional prior dynamics using a linear map. KoVAE outperforms state-of-the-art GAN and VAE methods across several challenging synthetic and real-world time series generation benchmarks.
  arXiv  Detail & Related papers  (2023-10-04T07:14:43Z)
• ChiroDiff: Modelling chirographic data with Diffusion Models [132.5223191478268]
  We introduce a powerful model-class namely "Denoising Diffusion Probabilistic Models" or DDPMs for chirographic data. Our model named "ChiroDiff", being non-autoregressive, learns to capture holistic concepts and therefore remains resilient to higher temporal sampling rate.
  arXiv  Detail & Related papers  (2023-04-07T15:17:48Z)
• What Are Bayesian Neural Network Posteriors Really Like? [63.950151520585024]
  We show that Hamiltonian Monte Carlo can achieve significant performance gains over standard and deep ensembles. We also show that deep distributions are similarly close to HMC as standard SGLD, and closer than standard variational inference.
  arXiv  Detail & Related papers  (2021-04-29T15:38:46Z)
• Learning representations with end-to-end models for improved remaining useful life prognostics [64.80885001058572]
  The remaining Useful Life (RUL) of equipment is defined as the duration between the current time and its failure. We propose an end-to-end deep learning model based on multi-layer perceptron and long short-term memory layers (LSTM) to predict the RUL. We will discuss how the proposed end-to-end model is able to achieve such good results and compare it to other deep learning and state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-04-11T16:45:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.