Monotonic Kolmogorov-Arnold Networks: A Theoretical and Empirical Study of Monotonicity as an Inductive Bias

• URL: http://arxiv.org/abs/2606.17886v2
• Date: Thu, 18 Jun 2026 07:32:54 GMT
• Title: Monotonic Kolmogorov-Arnold Networks: A Theoretical and Empirical Study of Monotonicity as an Inductive Bias
• Abstract summary: Monotonicity has been a long-running architectural inductive bias for neural networks.<n>We present bfMKAN, a KAN with hard monotonicity guaranteed for emphall parameter values.<n>MKAN is competitive with state-of-the-art monotone NNs on the SMM/I-2024 benchmark.
• Score: 4.904092547705666
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Monotonicity has been a long-running architectural inductive bias for neural networks, motivated by tabular, scientific, and economic settings where outputs are known to respond monotonically to certain inputs. Existing approaches are MLP- or flow-based and lack per-edge functional transparency; the only Kolmogorov--Arnold Network (KAN) variant with monotonicity, MonoKAN, enforces the constraint only on a restricted parameter subset and requires a projection-style training procedure. We close this gap with \textbf{MKAN}, a KAN with hard monotonicity guaranteed for \emph{all} parameter values via exponential reparameterization of B-spline coefficients, positive edge weights, and a monotone base activation. Training reduces to standard unconstrained gradient descent. Our headline theoretical contribution is a \emph{representation-cost} theorem: any $C^K, K >0$ feature extractor inducing a ball-shaped semantic-neighborhood partition admits a monotone realization of the equivalent neighborhood structure at $N' = N^* + k \le 2N^*$, where $k$ is the number of non-monotone coordinates of the original. The bound is architecture-agnostic and gives a principled sizing rule for monotone encoders. Empirically, MKAN is competitive with state-of-the-art monotone NNs on the SMM/ICML-2024 benchmark while being the only method that combines hard unconstrained monotonicity with KAN's per-edge functional transparency; the $2N^*$ prediction is validated in a self-supervised feature-size sweep on four real datasets, and on a controlled monotone-generative dataset MKAN recovers ground-truth factors with substantially higher Spearman alignment than KAN, MLP, and linear baselines.

Related papers

• Martingale Neural Operators: Learning Stochastic Marginals via Doob-Meyer Factorization [0.0]
  Doob-Meyer theorem establishes that any semimartingale fundamentally decomposes into a predictable drift and an unpredictable, zero-mean martingale.<n>We introduce the Martingale Neural Operator (MNO), which maps an initial condition directly to the conditional mean and covariance of the terminal law.<n>MNO reduces Wasserstein distance by up to $120times$ on $4$ field theory and $68times$ on Burgers, evaluating $sim 3times$ faster than a conditional diffusion baseline matched at wall-clock training budgets.
  arXiv  Detail & Related papers  (2026-05-15T10:00:21Z)
• Beyond the Laplacian: Doubly Stochastic Matrices for Graph Neural Networks [0.0]
  Graph Neural Networks (GNNs) conventionally rely on standard Laplacian or adjacency matrices for structural message passing.<n>In this work, we substitute the traditional Laplacian with a Doubly graph Matrix (DSM), derived from the inverse of the modified Laplacian, to naturally encode continuous multi-hop proximity and strict local centrality.
  arXiv  Detail & Related papers  (2026-04-16T14:33:32Z)
• Wiener Chaos Expansion based Neural Operator for Singular Stochastic Partial Differential Equations [58.188204772101756]
  This paper explores how our recently developed Wiener Chaos Expansion (WCE)-based neural operator (NO) can be applied to singular partial differential equations.<n>We leverage feature-wise linear modulation (FiLM) to appropriately capture the dependency between the solution of singular SPDE and its smooth remainder.<n>The resulting WCE-FiLM-NO shows excellent performance on $boldsymbol4$, as measured by relative $L$ loss, out-of-distribution and autocorrelation score.
  arXiv  Detail & Related papers  (2026-03-09T10:50:30Z)
• Regularized Online RLHF with Generalized Bilinear Preferences [68.44113000390544]
  We consider the problem of contextual online RLHF with general preferences.<n>We adopt the Generalized Bilinear Preference Model to capture preferences via low-rank, skew-symmetric matrices.<n>We prove that the dual gap of the greedy policy is bounded by the square of the estimation error.
  arXiv  Detail & Related papers  (2026-02-26T15:27:53Z)
• No More DeLuLu: Physics-Inspired Kernel Networks for Geometrically-Grounded Neural Computation [0.42303492200814446]
  We introduce the yat-product, a kernel operator combining quadratic alignment with inverse-square proximity.<n>We prove it is a Mercer kernel, analytic, Lipschitz on bounded domains, and self-regularizing.<n> Neural Matter Networks (NMNs) use yat-product as the sole non-linearity, replacing conventional linear-activation-normalization blocks with a single geometrically-grounded operation.
  arXiv  Detail & Related papers  (2026-02-22T01:09:31Z)
• Singular Bayesian Neural Networks [1.2891210250935148]
  Bayesian neural networks promise calibrated uncertainty but require $O(mn)$ parameters for standard mean-field Gaussian posteriors.<n>We induce a posterior that is singular with respect to the Lebesgue measure, concentrating on the rank-$r$ manifold.<n>We derive PAC-Bayes generalization bounds whose complexity term scales as $sqrtr(m+n)$ instead of $sqrtm n$, and prove loss bounds that decompose the error into optimization and rank-induced bias.
  arXiv  Detail & Related papers  (2026-01-30T23:06:34Z)
• Unregularized Linear Convergence in Zero-Sum Game from Preference Feedback [50.89125374999765]
  We provide the first convergence guarantee for Optimistic Multiplicative Weights Update ($mathtOMWU$) in NLHF.<n>Our analysis identifies a novel marginal convergence behavior, where the probability of rarely played actions grows exponentially from exponentially small values.
  arXiv  Detail & Related papers  (2025-12-31T12:08:29Z)
• Differentiable Sparsity via $D$-Gating: Simple and Versatile Structured Penalization [22.883367233817836]
  We show that $D$-Gating is theoretically equivalent to solving the original group sparsity problem.<n>We validate our theory across vision, language, and tasks, where $D$-Gating consistently delivers strong performance-sparsity tradeoffs.
  arXiv  Detail & Related papers  (2025-09-28T14:08:29Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Transformers as Support Vector Machines [54.642793677472724]
  We establish a formal equivalence between the optimization geometry of self-attention and a hard-margin SVM problem. We characterize the implicit bias of 1-layer transformers optimized with gradient descent. We believe these findings inspire the interpretation of transformers as a hierarchy of SVMs that separates and selects optimal tokens.
  arXiv  Detail & Related papers  (2023-08-31T17:57:50Z)
• Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approach [60.67748036747221]
  Implicit neural networks offer competitive performance and reduced memory consumption. They can remain brittle with respect to input adversarial perturbations. This paper proposes a theoretical and computational framework for robustness verification of implicit neural networks.
  arXiv  Detail & Related papers  (2021-12-10T03:08:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.