MLPs and KANs for data-driven learning in physical problems: A performance comparison

• URL: http://arxiv.org/abs/2504.11397v1
• Date: Tue, 15 Apr 2025 17:13:42 GMT
• Title: MLPs and KANs for data-driven learning in physical problems: A performance comparison
• Authors: Raghav Pant, Sikan Li, Xingjian Li, Hassan Iqbal, Krishna Kumar,
• Abstract summary: Kolmogorov-Layer Networks (KANs) are an alternative to traditional neural networks represented by Multi-Arnold Perceptrons (MLPs)<n>While showing promise, their performance advantages in physics-based problems remain largely unexplored.<n>This suggests that KANs are a promising choice, offering a balance of efficiency and accuracy in applications involving physical systems.
• Score: 4.252092276491948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is increasing interest in solving partial differential equations (PDEs) by casting them as machine learning problems. Recently, there has been a spike in exploring Kolmogorov-Arnold Networks (KANs) as an alternative to traditional neural networks represented by Multi-Layer Perceptrons (MLPs). While showing promise, their performance advantages in physics-based problems remain largely unexplored. Several critical questions persist: Can KANs capture complex physical dynamics and under what conditions might they outperform traditional architectures? In this work, we present a comparative study of KANs and MLPs for learning physical systems governed by PDEs. We assess their performance when applied in deep operator networks (DeepONet) and graph network-based simulators (GNS), and test them on physical problems that vary significantly in scale and complexity. Drawing inspiration from the Kolmogorov Representation Theorem, we examine the behavior of KANs and MLPs across shallow and deep network architectures. Our results reveal that although KANs do not consistently outperform MLPs when configured as deep neural networks, they demonstrate superior expressiveness in shallow network settings, significantly outpacing MLPs in accuracy over our test cases. This suggests that KANs are a promising choice, offering a balance of efficiency and accuracy in applications involving physical systems.

Related papers

• Enhancing Physics-Informed Neural Networks with a Hybrid Parallel Kolmogorov-Arnold and MLP Architecture [0.0]
  We propose a novel architecture that integrates parallelized KAN and branches within a unified PINN framework.<n>The HPKM-PINN introduces a scaling factor xi, to optimally balance the complementary strengths of KAN's interpretable function approximation and numerical's nonlinear learning.<n>These findings highlight the HPKM-PINN's ability to leverage KAN's interpretability and robustness, positioning it as a versatile and scalable tool for solving complex PDE-driven problems.
  arXiv  Detail & Related papers  (2025-03-30T02:59:32Z)
• SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networks [0.9999629695552196]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising method for solving partial differential equations (PDEs) We introduce Separable Physics-Informed Kolmogorov-Arnold Networks (SPIKANs) This novel architecture applies the principle of separation of variables to PIKANs, decomposing the problem such that each dimension is handled by an individual KAN.
  arXiv  Detail & Related papers  (2024-11-09T21:10:23Z)
• A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networks [43.70716358136333]
  Kolmogorov- Networks (KAN) are based on a fundamentally different mathematical framework. KANs address several major issues insio, such as forgetting in continual learning scenarios. We extend the investigation by evaluating the performance of KANs in continual learning tasks within computer vision.
  arXiv  Detail & Related papers  (2024-09-20T14:49:21Z)
• Physics Informed Kolmogorov-Arnold Neural Networks for Dynamical Analysis via Efficent-KAN and WAV-KAN [0.12045539806824918]
  We implement the Physics-Informed Kolmogorov-Arnold Neural Networks (PIKAN) through efficient-KAN and WAV-KAN. PIKAN demonstrates superior performance compared to conventional deep neural networks, achieving the same level of accuracy with fewer layers and reduced computational overhead.
  arXiv  Detail & Related papers  (2024-07-25T20:14:58Z)
• A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks [8.573300153709358]
  Kolmogorov-Arnold Networks (KANs) were recently introduced as an alternative representation model to the standard representation model. Herein, we employ KANs to construct machine learning models (PIKANs) and deep operator models (DeepOKANs) for solving differential equations for forward and inverse problems.
  arXiv  Detail & Related papers  (2024-06-05T04:10:36Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• How Neural Networks Extrapolate: From Feedforward to Graph Neural Networks [80.55378250013496]
  We study how neural networks trained by gradient descent extrapolate what they learn outside the support of the training distribution. Graph Neural Networks (GNNs) have shown some success in more complex tasks.
  arXiv  Detail & Related papers  (2020-09-24T17:48:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.