Enhancing Neural Function Approximation: The XNet Outperforming KAN

• URL: http://arxiv.org/abs/2501.18959v2
• Date: Fri, 14 Feb 2025 02:50:45 GMT
• Title: Enhancing Neural Function Approximation: The XNet Outperforming KAN
• Authors: Xin Li, Xiaotao Zheng, Zhihong Xia,
• Abstract summary: XNet is a single-layer neural network architecture that leverages Cauchy integral-based activation functions for high-order function.<n>We show that the Cauchy activation functions used in XNet can achieve arbitrary-order convergence.<n>Results establish XNet as a highly efficient architecture for both scientific computing and AI applications.
• Score: 3.9426000822656224
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: XNet is a single-layer neural network architecture that leverages Cauchy integral-based activation functions for high-order function approximation. Through theoretical analysis, we show that the Cauchy activation functions used in XNet can achieve arbitrary-order polynomial convergence, fundamentally outperforming traditional MLPs and Kolmogorov-Arnold Networks (KANs) that rely on increased depth or B-spline activations. Our extensive experiments on function approximation, PDE solving, and reinforcement learning demonstrate XNet's superior performance - reducing approximation error by up to 50000 times and accelerating training by up to 10 times compared to existing approaches. These results establish XNet as a highly efficient architecture for both scientific computing and AI applications.

Related papers

• Cauchy activation function and XNet [3.9426000822656224]
  We have developed a novel activation function, named the Cauchy Activation Function.<n>This function is derived from the Cauchy Integral Theorem in complex analysis and is specifically tailored for problems requiring high precision.<n>This innovation has led to the creation of a new class of neural networks, which we call (Comple)XNet, or simply XNet.
  arXiv  Detail & Related papers  (2024-09-28T03:25:33Z)
• Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation [0.0]
  This paper presents the Chebyshev Kolmogorov-Arnold Network (Chebyshev KAN), a new neural network architecture inspired by the Kolmogorov-Arnold theorem. By utilizing learnable functions parametrized by Chebyshevs on the network's edges, Chebyshev KANs enhance flexibility, efficiency, and interpretability in function approximation tasks.
  arXiv  Detail & Related papers  (2024-05-12T07:55:43Z)
• A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons and Adaptable Structure [29.11456970277094]
  We show that X-Net can achieve comparable or even better performance than neurons on regression and classification tasks. X-Net is shown to help scientists discover new laws of mathematics or physics.
  arXiv  Detail & Related papers  (2024-01-03T14:52:18Z)
• Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization [73.80101701431103]
  The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks. We study the usefulness of the LLA in Bayesian optimization and highlight its strong performance and flexibility.
  arXiv  Detail & Related papers  (2023-04-17T14:23:43Z)
• KXNet: A Model-Driven Deep Neural Network for Blind Super-Resolution [57.882146858582175]
  We propose a model-driven deep neural network, called KXNet, for blind SISR. The proposed KXNet is fully integrated with the inherent physical mechanism underlying this SISR task. Experiments on synthetic and real data finely demonstrate the superior accuracy and generality of our method.
  arXiv  Detail & Related papers  (2022-09-21T12:22:50Z)
• Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach [64.23331120621118]
  This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks. We introduce a related embedded network and show that the embedded network can be used to provide an $ell_infty$-norm box over-approximation of the reachable sets of the original network. We apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.
  arXiv  Detail & Related papers  (2022-08-08T03:13:24Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Reachability analysis of neural networks using mixed monotonicity [0.0]
  We present a new reachability analysis tool to compute an interval over-approximation of the output set of a feedforward neural network under given input uncertainty. The proposed approach adapts to neural networks an existing mixed-monotonicity method for the reachability analysis of dynamical systems.
  arXiv  Detail & Related papers  (2021-11-15T11:35:18Z)
• Going Beyond Linear RL: Sample Efficient Neural Function Approximation [76.57464214864756]
  We study function approximation with two-layer neural networks. Our results significantly improve upon what can be attained with linear (or eluder dimension) methods.
  arXiv  Detail & Related papers  (2021-07-14T03:03:56Z)
• Fully-parallel Convolutional Neural Network Hardware [0.7829352305480285]
  We propose a new power-and-area-efficient architecture for implementing Articial Neural Networks (ANNs) in hardware. For the first time, a fully-parallel CNN as LENET-5 is embedded and tested in a single FPGA.
  arXiv  Detail & Related papers  (2020-06-22T17:19:09Z)
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks [50.42141893913188]
  We study a distributed variable for large-scale AUC for a neural network as with a deep neural network. Our model requires a much less number of communication rounds and still a number of communication rounds in theory. Our experiments on several datasets show the effectiveness of our theory and also confirm our theory.
  arXiv  Detail & Related papers  (2020-05-05T18:08:23Z)
• ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units via Chebyshev Approximations [6.0889567811100385]
  We propose a new and more stable way to construct RePU deep neural networks based on Chebyshev approximations. The approximation of smooth functions by ChebNets is no worse than the approximation by deep RePU nets using power series approach.
  arXiv  Detail & Related papers  (2019-11-07T06:30:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.