Scaling of Gaussian Kolmogorov--Arnold Networks

• URL: http://arxiv.org/abs/2604.21174v1
• Date: Thu, 23 Apr 2026 00:41:21 GMT
• Title: Scaling of Gaussian Kolmogorov--Arnold Networks
• Abstract summary: We investigate how () affects Gaussian KANs through first-layer feature geometry, conditioning, and approximation behavior.<n>We show that this range is useful for fixed-scale selection, variable-scale constructions, constrained training of () and efficient scale search using early training MSE.
• Score: 2.589700903743416
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Gaussian scale parameter \(ε\) is central to the behavior of Gaussian Kolmogorov--Arnold Networks (KANs), yet its role in deep edge-based architectures has not been studied systematically. In this paper, we investigate how \(ε\) affects Gaussian KANs through first-layer feature geometry, conditioning, and approximation behavior. Our central observation is that scale selection is governed primarily by the first layer, since it is the only layer constructed directly on the input domain and any loss of distinguishability introduced there cannot be recovered by later layers. From this viewpoint, we analyze the first-layer feature matrix and identify a practical operating interval, \[ ε\in \left[\frac{1}{G-1},\frac{2}{G-1}\right], \] where \(G\) denotes the number of Gaussian centers. For the standard shared-center Gaussian KAN used in current practice, we interpret this interval not as a universal optimality result, but as a stable and effective design rule, and validate it through brute-force sweeps over \(ε\) across function-approximation problems with different collocation densities, grid resolutions, network architectures, and input dimensions, as well as a physics-informed Helmholtz problem. We further show that this range is useful for fixed-scale selection, variable-scale constructions, constrained training of \(ε\), and efficient scale search using early training MSE. Finally, using a matched Chebyshev reference, we show that a properly scaled Gaussian KAN can already be competitive in accuracy relative to another standard KAN basis. In this way, the paper positions scale selection as a practical design principle for Gaussian KANs rather than as an ad hoc hyperparameter choice.

Related papers

• Partition-of-Unity Gaussian Kolmogorov-Arnold Networks [0.0]
  We introduce the partition-of-unity Gaussian KAN (PU-GKAN) in which the Gaussian basis values on each edge are divided by their local sum over fixed centers.<n>This produces a partition-of-unity feature map with trainable coefficients, while preserving the standard edge-based KAN structure.<n> Numerical experiments show that PU-GKAN reduces sensitivity to (), improves validation accuracy for most smooth and moderately non-smooth targets, and gives more stable training behavior.
  arXiv  Detail & Related papers  (2026-04-26T08:24:03Z)
• Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learning [0.0]
  Internal noise in deep networks is usually inherited from perturbations such as dropout, hard masking, or additive perturbations.<n>We ask two questions: what correlation geometry should internal noise have, and is the implemented perturbation compatible with the representations it acts on?<n>We answer these questions through Variational Kernel Design (VKD), a framework in which a noise mechanism is specified by a law family, a correlation kernel, and an injection operator, and is derived from learning desiderata.
  arXiv  Detail & Related papers  (2026-03-18T05:12:56Z)
• K-DAREK: Distance Aware Error for Kurkova Kolmogorov Networks [3.460138063155115]
  We develop a novel learning algorithm, distance-aware error for Kurkova-Kolmogorov networks (K-DAREK), for efficient and interpretable function approximation with uncertainty quantification.<n>Our approach establishes robust error bounds that are distance-aware; this means they reflect the proximity of a test point to its nearest training points.
  arXiv  Detail & Related papers  (2025-10-24T20:49:59Z)
• Gaussian Primitive Optimized Deformable Retinal Image Registration [19.882820812725523]
  Deformable retinal image registration is notoriously difficult due to large homogeneous regions and sparse but critical vascular features.<n>We introduce a novel iterative framework that performs structured message passing to overcome these challenges.<n>Experiments on the FIRE dataset show that GPO reduces the target registration error from 6.2,px to 2.4,px and increases the AUC at 25,px from 0.770 to 0.938.
  arXiv  Detail & Related papers  (2025-08-23T00:44:50Z)
• Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks [4.554284689395686]
  implicit gradient descent (IGD) outperforms the common gradient descent (GD) algorithm in handling certain multi-scale problems.<n>We show that IGD converges to a globally optimal solution at a linear convergence rate.
  arXiv  Detail & Related papers  (2024-07-03T06:10:41Z)
• Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization [73.80101701431103]
  The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks. We study the usefulness of the LLA in Bayesian optimization and highlight its strong performance and flexibility.
  arXiv  Detail & Related papers  (2023-04-17T14:23:43Z)
• On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks [21.69222364939501]
  In supervised learning, the regularization path is sometimes used as a convenient theoretical proxy for the optimization path of gradient descent from zero. We show that the path interpolates continuously between the so-called kernel and rich regimes.
  arXiv  Detail & Related papers  (2023-03-31T05:32:11Z)
• NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer [45.47667026025716]
  We propose a novel, robust and accelerated iteration that relies on two key elements. The convergence and stability of the obtained method, referred to as NAG-GS, are first studied extensively. We show that NAG-arity is competitive with state-the-art methods such as momentum SGD with weight decay and AdamW for the training of machine learning models.
  arXiv  Detail & Related papers  (2022-09-29T16:54:53Z)
• Gaussian Kernel Variance For an Adaptive Learning Method on Signals Over Graphs [10.028519427235326]
  Single- Kernel Gradraker (SKG) is an adaptive learning method predicting unknown nodal values in a network. We focus on SKG with a Gaussian kernel and specify how to find a suitable variance for the kernel.
  arXiv  Detail & Related papers  (2022-04-26T23:15:03Z)
• What Are Bayesian Neural Network Posteriors Really Like? [63.950151520585024]
  We show that Hamiltonian Monte Carlo can achieve significant performance gains over standard and deep ensembles. We also show that deep distributions are similarly close to HMC as standard SGLD, and closer than standard variational inference.
  arXiv  Detail & Related papers  (2021-04-29T15:38:46Z)
• Improving predictions of Bayesian neural nets via local linearization [79.21517734364093]
  We argue that the Gauss-Newton approximation should be understood as a local linearization of the underlying Bayesian neural network (BNN) Because we use this linearized model for posterior inference, we should also predict using this modified model instead of the original one. We refer to this modified predictive as "GLM predictive" and show that it effectively resolves common underfitting problems of the Laplace approximation.
  arXiv  Detail & Related papers  (2020-08-19T12:35:55Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)
• Revisiting Initialization of Neural Networks [72.24615341588846]
  We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
  arXiv  Detail & Related papers  (2020-04-20T18:12:56Z)
• Gradient Centralization: A New Optimization Technique for Deep Neural Networks [74.935141515523]
  gradient centralization (GC) operates directly on gradients by centralizing the gradient vectors to have zero mean. GC can be viewed as a projected gradient descent method with a constrained loss function. GC is very simple to implement and can be easily embedded into existing gradient based DNNs with only one line of code.
  arXiv  Detail & Related papers  (2020-04-03T10:25:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.