Learning Nonlinear Heterogeneity in Physical Kolmogorov-Arnold Networks

• URL: http://arxiv.org/abs/2601.15340v2
• Date: Sun, 25 Jan 2026 13:21:22 GMT
• Title: Learning Nonlinear Heterogeneity in Physical Kolmogorov-Arnold Networks
• Authors: Fabiana Taglietti, Andrea Pulici, Maxwell Roxburgh, Gabriele Seguini, Ian Vidamour, Stephan Menzel, Edoardo Franco, Michele Laus, Eleni Vasilaki, Michele Perego, Thomas J. Hayward, Marco Fanciulli, Jack C. Gartside,
• Abstract summary: Physical neural networks typically train linear synaptic weights while treating device nonlinearities as fixed.<n>We show the opposite - by training the synaptic nonlinearity itself, we yield markedly higher task performance per physical resource.<n>We experimentally realise physical KANs in silicon-on-insulator devices we term 'Synaptic Elements'
• Score: 0.7743990032107954
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Physical neural networks typically train linear synaptic weights while treating device nonlinearities as fixed. We show the opposite - by training the synaptic nonlinearity itself, as in Kolmogorov-Arnold Network (KAN) architectures, we yield markedly higher task performance per physical resource and improved performance-parameter scaling than conventional linear weight-based networks, demonstrating ability of KAN topologies to exploit reconfigurable nonlinear physical dynamics. We experimentally realise physical KANs in silicon-on-insulator devices we term 'Synaptic Nonlinear Elements' (SYNEs), operating at room temperature, microampere currents, 2 MHz speeds and ~250 fJ per nonlinear operation, with no observed degradation over 10^13 measurements and months-long timescales. We demonstrate nonlinear function regression, classification, and prediction of Li-Ion battery dynamics from noisy real-world multi-sensor data. Physical KANs outperform equivalently-parameterised software multilayer perceptron networks across all tasks, with up to two orders of magnitude fewer parameters, and two orders of magnitude fewer devices than linear weight based physical networks. These results establish learned physical nonlinearity as a hardware-native computational primitive for compact and efficient learning systems, and SYNE devices as effective substrates for heterogenous nonlinear computing.

Related papers

• Bridging Function Approximation and Device Physics via Negative Differential Resistance Networks [17.356504841908826]
  KANalogue is a fully analogue implementation of Kolmogorov-Arnold Networks (KANs) using negative differential resistance devices.<n>By leveraging the intrinsic negative differential resistance characteristics of tunnel diodes fabricated from NbSi2N4/HfSi2N4 heterostructures, we construct coordinate-wise nonlinearities.<n>Our results demonstrate that KANalogue can approximate complex functions with minimal parameters while maintaining classification accuracy competitive with digital baselines.
  arXiv  Detail & Related papers  (2025-10-24T15:38:22Z)
• Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models [51.85815025140659]
  Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data.<n>In particular, the proportional regime where the data dimension, sample size, and number of model parameters are all large gives rise to novel and sometimes counterintuitive behaviors.<n>This paper extends traditional Random Matrix Theory (RMT) beyond eigenvalue-based analysis of linear models to address the challenges posed by nonlinear ML models.
  arXiv  Detail & Related papers  (2025-06-16T06:54:08Z)
• Neural Contraction Metrics with Formal Guarantees for Discrete-Time Nonlinear Dynamical Systems [17.905596843865705]
  Contraction metrics provide a powerful framework for analyzing stability, robustness, and convergence of various dynamical systems.<n>However, identifying these metrics for complex nonlinear systems remains an open challenge due to the lack of effective tools.<n>This paper develops verifiable contraction metrics for discrete scalable nonlinear systems.
  arXiv  Detail & Related papers  (2025-04-23T21:27:32Z)
• Generative System Dynamics in Recurrent Neural Networks [56.958984970518564]
  We investigate the continuous time dynamics of Recurrent Neural Networks (RNNs)<n>We show that skew-symmetric weight matrices are fundamental to enable stable limit cycles in both linear and nonlinear configurations.<n> Numerical simulations showcase how nonlinear activation functions not only maintain limit cycles, but also enhance the numerical stability of the system integration process.
  arXiv  Detail & Related papers  (2025-04-16T10:39:43Z)
• Multiscale Analysis of Woven Composites Using Hierarchical Physically Recurrent Neural Networks [0.0]
  Multiscale homogenization of woven composites requires detailed micromechanical evaluations.<n>This study introduces a Hierarchical Physically Recurrent Neural Network (HPRNN) employing two levels of surrogate modeling.
  arXiv  Detail & Related papers  (2025-03-06T19:02:32Z)
• Physics-Informed Regularization for Domain-Agnostic Dynamical System Modeling [41.82469276824927]
  We present a framework that achieves high-precision modeling for a wide range of dynamical systems. It helps preserve energies for conservative systems while serving as a strong inductive bias for non-conservative, reversible systems. By integrating the TRS loss within neural ordinary differential equation models, the proposed model TREAT demonstrates superior performance on diverse physical systems.
  arXiv  Detail & Related papers  (2024-10-08T21:04:01Z)
• TANGO: Time-Reversal Latent GraphODE for Multi-Agent Dynamical Systems [43.39754726042369]
  We propose a simple-yet-effective self-supervised regularization term as a soft constraint that aligns the forward and backward trajectories predicted by a continuous graph neural network-based ordinary differential equation (GraphODE) It effectively imposes time-reversal symmetry to enable more accurate model predictions across a wider range of dynamical systems under classical mechanics. Experimental results on a variety of physical systems demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2023-10-10T08:52:16Z)
• Physics-Informed Long Short-Term Memory for Forecasting and Reconstruction of Chaos [5.8010446129208155]
  We present the Physics-Informed Long Short-Term Memory (PI-LSTM) network to reconstruct and predict the evolution of unmeasured variables in a chaotic system. The training is constrained by a regularization term, which penalizes solutions that violate the system's governing equations. This work opens up new opportunities for state reconstruction and learning of the dynamics of nonlinear systems.
  arXiv  Detail & Related papers  (2023-02-03T18:27:59Z)
• Physics-informed Neural Network for Nonlinear Dynamics in Fiber Optics [10.335960060544904]
  A physics-informed neural network (PINN) that combines deep learning with physics is studied to solve the nonlinear Schr"odinger equation for learning nonlinear dynamics in fiber optics. PINN is not only an effective partial differential equation solver, but also a prospective technique to advance the scientific computing and automatic modeling in fiber optics.
  arXiv  Detail & Related papers  (2021-09-01T12:19:32Z)
• Linear embedding of nonlinear dynamical systems and prospects for efficient quantum algorithms [74.17312533172291]
  We describe a method for mapping any finite nonlinear dynamical system to an infinite linear dynamical system (embedding) We then explore an approach for approximating the resulting infinite linear system with finite linear systems (truncation)
  arXiv  Detail & Related papers  (2020-12-12T00:01:10Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)
• Liquid Time-constant Networks [117.57116214802504]
  We introduce a new class of time-continuous recurrent neural network models. Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems. These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
  arXiv  Detail & Related papers  (2020-06-08T09:53:35Z)
• Training End-to-End Analog Neural Networks with Equilibrium Propagation [64.0476282000118]
  We introduce a principled method to train end-to-end analog neural networks by gradient descent. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
  arXiv  Detail & Related papers  (2020-06-02T23:38:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.