Opening the Black-Box: Symbolic Regression with Kolmogorov-Arnold Networks for Energy Applications

• URL: http://arxiv.org/abs/2504.03913v1
• Date: Fri, 04 Apr 2025 20:23:33 GMT
• Title: Opening the Black-Box: Symbolic Regression with Kolmogorov-Arnold Networks for Energy Applications
• Authors: Nataly R. Panczyk, Omer F. Erdem, Majdi I. Radaideh,
• Abstract summary: This work compares a traditional feedforward neural network (FNN) to a Kolmogorov-Arnold Network (KAN)<n>In terms of accuracy, we find KANs and FNNs comparable across all datasets, when output dimensionality is limited.<n>KANs, which transform into symbolic equations after training, yield perfectly interpretable models while FNNs remain black-boxes.
• Score: 0.44241702149260353
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: While most modern machine learning methods offer speed and accuracy, few promise interpretability or explainability -- two key features necessary for highly sensitive industries, like medicine, finance, and engineering. Using eight datasets representative of one especially sensitive industry, nuclear power, this work compares a traditional feedforward neural network (FNN) to a Kolmogorov-Arnold Network (KAN). We consider not only model performance and accuracy, but also interpretability through model architecture and explainability through a post-hoc SHAP analysis. In terms of accuracy, we find KANs and FNNs comparable across all datasets, when output dimensionality is limited. KANs, which transform into symbolic equations after training, yield perfectly interpretable models while FNNs remain black-boxes. Finally, using the post-hoc explainability results from Kernel SHAP, we find that KANs learn real, physical relations from experimental data, while FNNs simply produce statistically accurate results. Overall, this analysis finds KANs a promising alternative to traditional machine learning methods, particularly in applications requiring both accuracy and comprehensibility.

Related papers

• GINN-KAN: Interpretability pipelining with applications in Physics Informed Neural Networks [5.2969467015867915]
  We introduce the concept of interpretability pipelineing, to incorporate multiple interpretability techniques to outperform each individual technique. We evaluate two recent models selected for their potential to incorporate interpretability into standard neural network architectures. We introduce a novel interpretable neural network GINN-KAN that synthesizes the advantages of both models.
  arXiv  Detail & Related papers  (2024-08-27T04:57:53Z)
• Physics Informed Kolmogorov-Arnold Neural Networks for Dynamical Analysis via Efficent-KAN and WAV-KAN [0.12045539806824918]
  We implement the Physics-Informed Kolmogorov-Arnold Neural Networks (PIKAN) through efficient-KAN and WAV-KAN. PIKAN demonstrates superior performance compared to conventional deep neural networks, achieving the same level of accuracy with fewer layers and reduced computational overhead.
  arXiv  Detail & Related papers  (2024-07-25T20:14:58Z)
• Bayesian Entropy Neural Networks for Physics-Aware Prediction [14.705526856205454]
  We introduce BENN, a framework designed to impose constraints on Bayesian Neural Network (BNN) predictions. Benn is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. Results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.
  arXiv  Detail & Related papers  (2024-07-01T07:00:44Z)
• Contextualizing MLP-Mixers Spatiotemporally for Urban Data Forecast at Scale [54.15522908057831]
  We propose an adapted version of the computationally-Mixer for STTD forecast at scale. Our results surprisingly show that this simple-yeteffective solution can rival SOTA baselines when tested on several traffic benchmarks. Our findings contribute to the exploration of simple-yet-effective models for real-world STTD forecasting.
  arXiv  Detail & Related papers  (2023-07-04T05:19:19Z)
• Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection [55.028065567756066]
  Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem. We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
  arXiv  Detail & Related papers  (2022-06-26T16:00:22Z)
• Knowledge Enhanced Neural Networks for relational domains [83.9217787335878]
  We focus on a specific method, KENN, a Neural-Symbolic architecture that injects prior logical knowledge into a neural network. In this paper, we propose an extension of KENN for relational data.
  arXiv  Detail & Related papers  (2022-05-31T13:00:34Z)
• MGDCF: Distance Learning via Markov Graph Diffusion for Neural Collaborative Filtering [96.65234340724237]
  We show the equivalence between some state-of-the-art GNN-based CF models and a traditional 1-layer NRL model based on context encoding. We present Markov Graph Diffusion Collaborative Filtering (MGDCF) to generalize some state-of-the-art GNN-based CF models.
  arXiv  Detail & Related papers  (2022-04-05T17:24:32Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Neural Networks Enhancement with Logical Knowledge [83.9217787335878]
  We propose an extension of KENN for relational data. The results show that KENN is capable of increasing the performances of the underlying neural network even in the presence relational data.
  arXiv  Detail & Related papers  (2020-09-13T21:12:20Z)
• Neural Additive Models: Interpretable Machine Learning with Neural Nets [77.66871378302774]
  Deep neural networks (DNNs) are powerful black-box predictors that have achieved impressive performance on a wide variety of tasks. We propose Neural Additive Models (NAMs) which combine some of the expressivity of DNNs with the inherent intelligibility of generalized additive models. NAMs learn a linear combination of neural networks that each attend to a single input feature.
  arXiv  Detail & Related papers  (2020-04-29T01:28:32Z)
• Adaptive Explainable Neural Networks (AxNNs) [8.949704905866888]
  We develop a new framework called Adaptive Explainable Neural Networks (AxNN) for achieving the dual goals of good predictive performance and model interpretability. For predictive performance, we build a structured neural network made up of ensembles of generalized additive model networks and additive index models. For interpretability, we show how to decompose the results of AxNN into main effects and higher-order interaction effects.
  arXiv  Detail & Related papers  (2020-04-05T23:40:57Z)
• A deep learning framework for solution and discovery in solid mechanics [1.4699455652461721]
  We present the application of a class of deep learning, known as Physics Informed Neural Networks (PINN), to learning and discovery in solid mechanics. We explain how to incorporate the momentum balance and elasticity relations into PINN, and explore in detail the application to linear elasticity.
  arXiv  Detail & Related papers  (2020-02-14T08:24:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.