SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits

• URL: http://arxiv.org/abs/2608.00859v1
• Date: Sat, 01 Aug 2026 20:42:41 GMT
• Title: SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits
• Abstract summary: We present textbfSparseKAN, a unified approach that compresses KANs along three complementary axes: basis functions, neurons/channels, and numerical precision.<n>SparseKAN equips the base branch, nonlinear basis branch, and individual basis terms with hierarchical learnable gates trained under a differentiable active-cost objective.
• Score: 6.860988566886594
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Kolmogorov--Arnold Networks (KANs) replace scalar edge weights with learnable univariate functions parameterized by multiple basis coefficients. This introduces a source of redundancy that conventional neural-network compression does not directly expose. We present \textbf{SparseKAN}, a unified approach that compresses KANs along three complementary axes: basis functions, neurons/channels, and numerical precision. SparseKAN equips the base branch, nonlinear basis branch, and individual basis terms with hierarchical learnable gates trained under a differentiable active-cost objective. The learned importance structure is subsequently hardened under explicit basis and width budgets, recovered in full or low precision, and physically compacted into smaller dense tensors rather than retained as sparse masks. Experiments on MNIST, CIFAR-10, and CIFAR-100 across spline, polynomial, RBF, wavelet, and convolutional KAN variants show that the structural axes compose predictably in cost. We also find strong basis-dependent differences in term importance: coefficient-based selection outperforms matched low-order truncation by up to 15.25 accuracy points in the evaluated Gram-polynomial settings. Eight-bit quantization is broadly robust, whereas 4-bit convolutional KANs require quantization-aware adaptation. Physical compaction removes up to 73.0\% of parameters without accuracy loss on MNIST and reduces large-batch CUDA latency to as little as $0.51\times$ dense execution. On a ZCU104 FPGA, the resulting sparse low-bit models achieve up to $23.63\times$ lower inference latency, demonstrating that SparseKAN converts functional redundancy into measurable software and hardware efficiency. The SparseKAN implementation is available at https://github.com/OSU-STARLAB/SparseKAN.

Related papers

• An Embedded RISC-V Evaluation of Kolmogorov--Arnold Networks in Hard-Constrained Recurrent Physics-Informed Models [0.0]
  Kolmogorov--Arnold Networks (KANs) have been proposed as efficient replacements for multilayer perceptrons (MLPs)<n>This paper asks whether that parameter efficiency survives deployment.
  arXiv  Detail & Related papers  (2026-08-01T16:20:24Z)
• LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models [56.999481798138625]
  LimiX-2M is a 2M- parameter model that outperforms larger TabPFN-v2 and TabICL baselines on widely used benchmarks.<n>We present a unified tokenize-and-route framework for strong Tabular foundation models (TFMs)<n>Results highlight value-aware tokenization and readout-aligned routing as key levers for improving the accuracy--efficiency trade-off in TFMs.
  arXiv  Detail & Related papers  (2026-06-03T06:07:33Z)
• KANtize: Exploring Low-bit Quantization of Kolmogorov-Arnold Networks for Efficient Inference [1.5102260054654923]
  Kolmogorov-Arnold Networks (KANs) have gained attention for their potential to outperform Multi-Layer Perceptrons (MLPs)<n>This study investigates the impact of low-bit quantization on KANs and its impact on computational complexity and hardware efficiency.
  arXiv  Detail & Related papers  (2026-03-18T00:32:11Z)
• A Quantization-Aware Training Based Lightweight Method for Neural Distinguishers [4.751864584128287]
  In 2019, Gohr pioneered the application of deep neural networks to differential cryptanalysis.<n>This study proposes a lightweight neural distinguisher based on quantization-aware training.
  arXiv  Detail & Related papers  (2026-03-06T00:52:49Z)
• BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models [56.504879072674015]
  We propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients.<n>BPDQ enables serving Qwen2.5-72B on a single GTX 3090 with 83.85% GSM8K accuracy (vs. 90.83% at 16-bit)
  arXiv  Detail & Related papers  (2026-02-04T02:54:37Z)
• QuantKAN: A Unified Quantization Framework for Kolmogorov Arnold Networks [6.860988566886594]
  Kolmogorov Arnold Networks (KANs) replace linear transformations with spline-based function approximations distributed along network edges.<n>KANs offer strong expressivity and interpretability, but their heterogeneous spline and base branch parameters hinder efficient quantization.<n>We present QuantKAN, a unified framework for quantizing KANs across both quantization aware training (QAT) and post-training quantization regimes.
  arXiv  Detail & Related papers  (2025-11-24T02:05:16Z)
• MetaCluster: Enabling Deep Compression of Kolmogorov-Arnold Network [8.780976521229741]
  Kolmogorov-Arnold Networks (KANs) replace scalar weights with per-edge vectors of basis coefficients.<n>We propose MetaCluster, a framework that makes KANs highly compressible without sacrificing accuracy.
  arXiv  Detail & Related papers  (2025-10-21T21:58:15Z)
• Incorporating Arbitrary Matrix Group Equivariance into KANs [69.30866522377694]
  Kolmogorov-Arnold Networks (KANs) have seen great success in scientific domains.<n>We propose Equivariant Kolmogorov-Arnold Networks (EKAN) to broaden their applicability to more fields.
  arXiv  Detail & Related papers  (2024-10-01T06:34:58Z)
• Compact representations of convolutional neural networks via weight pruning and quantization [63.417651529192014]
  We propose a novel storage format for convolutional neural networks (CNNs) based on source coding and leveraging both weight pruning and quantization. We achieve a reduction of space occupancy up to 0.6% on fully connected layers and 5.44% on the whole network, while performing at least as competitive as the baseline.
  arXiv  Detail & Related papers  (2021-08-28T20:39:54Z)
• Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression [56.83861738731913]
  Deep Neural Network (DNN) models are essential for practical applications, especially for resource limited devices. Previous unstructured or structured weight pruning methods can hardly truly accelerate inference. We propose a generalized weight unification framework at a hardware compatible micro-structured level to achieve high amount of compression and acceleration.
  arXiv  Detail & Related papers  (2021-06-15T17:22:59Z)
• Robust Implicit Networks via Non-Euclidean Contractions [63.91638306025768]
  Implicit neural networks show improved accuracy and significant reduction in memory consumption. They can suffer from ill-posedness and convergence instability. This paper provides a new framework to design well-posed and robust implicit neural networks.
  arXiv  Detail & Related papers  (2021-06-06T18:05:02Z)
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution [57.635467829558664]
  We introduce a structural regularization across convolutional kernels in a CNN. We show that CNNs now maintain performance with dramatic reduction in parameters and computations.
  arXiv  Detail & Related papers  (2020-09-04T20:41:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.