Low-power analogue neural networks with trainable nonlinear connections for continuous control

• URL: http://arxiv.org/abs/2606.23742v1
• Date: Sun, 21 Jun 2026 15:55:23 GMT
• Title: Low-power analogue neural networks with trainable nonlinear connections for continuous control
• Abstract summary: Physical neural networks promise low-power machine learning by computing directly with analogue device physics.<n>We place trainable nonlinear functions on the connections, making each physical connection a learnable computational element.<n>We find that the benefit is task-dependent and follows from the smoothness of the physical basis.
• Score: 54.59241649882409
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physical neural networks promise low-power machine learning by computing directly with analogue device physics, but most architectures force nonlinear device responses to act as scalar weights. Inspired by Kolmogorov-Arnold networks, we place trainable nonlinear functions on the connections, making each physical connection a learnable computational element. Realising these functions as analogue band-pass filters on field-programmable analogue arrays, we find that the benefit is task-dependent and follows from the smoothness of the physical basis: the networks represent smooth, continuously valued targets, including robotic kinematics, continuous control, and photovoltaic maximum-power-point tracking, with far fewer nodes and connections than multilayer perceptrons, but offer no parameter-efficiency advantage on classification-like decision boundaries. Trained networks transfer to hardware across approximately 35,000 connections with quantified fidelity, and a dedicated CMOS implementation is projected to operate at approximately 30 microwatts. A memristive realisation reproduces the same behaviour in simulation, indicating that the advantage comes from placing trainable nonlinearity on connections, rather than from a particular device.

Related papers

• Multi-layer MIMO Relay as Deep Physical Neural Networks: Power Amplifiers as Activation Functions [56.870614369768354]
  Wireless physical neural networks (WPNNs) embed neural computation directly into analog hardware.<n>We propose a deep WPNN in which nonlinear activations are realized by a multi-hop multiple-input multiple-output (MIMO) relay network.<n>We show that the proposed architecture enables accurate over-the-air inference for image classification.
  arXiv  Detail & Related papers  (2026-07-20T09:45:07Z)
• Relay-Assisted Activation-Integrated SIM for Wireless Physical Neural Networks [53.64687377045908]
  Wireless physical neural networks (WPNNs) have emerged as a promising paradigm for performing neural computation directly in the physical layer of wireless systems.<n>We propose a relay-assisted WPNN architecture based on activation-integrated stacked intelligent metasurfaces (AI-SIMs)
  arXiv  Detail & Related papers  (2026-04-05T18:07:13Z)
• Physical Analog Kolmogorov-Arnold Networks based on Reconfigurable Nonlinear-Processing Units [0.0]
  Kolmogorov-Arnold Networks (KANs) shift neural computation from linear layers to learnable nonlinear edge functions.<n>Here we introduce a physical analog KAN architecture in which edge functions are realized in materia using reconfigurable nonlinear-processing units (RNPUs)<n>We establish a realistic system-level hardware implementation that enables compact KAN-style regression and classification with programmable nonlinear transformations.
  arXiv  Detail & Related papers  (2026-02-07T12:33:11Z)
• Learning Nonlinear Heterogeneity in Physical Kolmogorov-Arnold Networks [0.7743990032107954]
  Physical neural networks typically train linear synaptic weights while treating device nonlinearities as fixed.<n>We show the opposite - by training the synaptic nonlinearity itself, we yield markedly higher task performance per physical resource.<n>We experimentally realise physical KANs in silicon-on-insulator devices we term 'Synaptic Elements'
  arXiv  Detail & Related papers  (2026-01-20T16:00:26Z)
• Bridging Function Approximation and Device Physics via Negative Differential Resistance Networks [17.356504841908826]
  KANalogue is a fully analogue implementation of Kolmogorov-Arnold Networks (KANs) using negative differential resistance devices.<n>By leveraging the intrinsic negative differential resistance characteristics of tunnel diodes fabricated from NbSi2N4/HfSi2N4 heterostructures, we construct coordinate-wise nonlinearities.<n>Our results demonstrate that KANalogue can approximate complex functions with minimal parameters while maintaining classification accuracy competitive with digital baselines.
  arXiv  Detail & Related papers  (2025-10-24T15:38:22Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers [105.74546828182834]
  We show a hardware-efficient dynamic inference regime, named dynamic weight slicing, which adaptively slice a part of network parameters for inputs with diverse difficulty levels. We present dynamic slimmable network (DS-Net) and dynamic slice-able network (DS-Net++) by input-dependently adjusting filter numbers of CNNs and multiple dimensions in both CNNs and transformers.
  arXiv  Detail & Related papers  (2021-09-21T09:57:21Z)
• Nonlinear computations in spiking neural networks through multiplicative synapses [3.1498833540989413]
  nonlinear computations can be implemented successfully in spiking neural networks. This requires supervised training and the resulting connectivity can be hard to interpret. We show how to directly derive the required connectivity for several nonlinear dynamical systems.
  arXiv  Detail & Related papers  (2020-09-08T16:47:27Z)
• Training End-to-End Analog Neural Networks with Equilibrium Propagation [64.0476282000118]
  We introduce a principled method to train end-to-end analog neural networks by gradient descent. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
  arXiv  Detail & Related papers  (2020-06-02T23:38:35Z)
• One-step regression and classification with crosspoint resistive memory arrays [62.997667081978825]
  High speed, low energy computing machines are in demand to enable real-time artificial intelligence at the edge. One-step learning is supported by simulations of the prediction of the cost of a house in Boston and the training of a 2-layer neural network for MNIST digit recognition. Results are all obtained in one computational step, thanks to the physical, parallel, and analog computing within the crosspoint array.
  arXiv  Detail & Related papers  (2020-05-05T08:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.