Universality of physical neural networks with multivariate nonlinearity

• URL: http://arxiv.org/abs/2509.05420v1
• Date: Sat, 06 Sep 2025 14:08:32 GMT
• Title: Universality of physical neural networks with multivariate nonlinearity
• Authors: Benjamin Savinson, David J. Norris, Siddhartha Mishra, Samuel Lanthaler,
• Abstract summary: Physical neural networks try to exploit physical systems to perform machine learning more efficiently.<n>We present a fundamental theorem that establishes a universality condition for physical neural networks.<n>We propose a scalable architecture using free-space optics that is provably universal and achieves high accuracy on image classification tasks.
• Score: 17.32613615949005
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The enormous energy demand of artificial intelligence is driving the development of alternative hardware for deep learning. Physical neural networks try to exploit physical systems to perform machine learning more efficiently. In particular, optical systems can calculate with light using negligible energy. While their computational capabilities were long limited by the linearity of optical materials, nonlinear computations have recently been demonstrated through modified input encoding. Despite this breakthrough, our inability to determine if physical neural networks can learn arbitrary relationships between data -- a key requirement for deep learning known as universality -- hinders further progress. Here we present a fundamental theorem that establishes a universality condition for physical neural networks. It provides a powerful mathematical criterion that imposes device constraints, detailing how inputs should be encoded in the tunable parameters of the physical system. Based on this result, we propose a scalable architecture using free-space optics that is provably universal and achieves high accuracy on image classification tasks. Further, by combining the theorem with temporal multiplexing, we present a route to potentially huge effective system sizes in highly practical but poorly scalable on-chip photonic devices. Our theorem and scaling methods apply beyond optical systems and inform the design of a wide class of universal, energy-efficient physical neural networks, justifying further efforts in their development.

Related papers

• Online unsupervised Hebbian learning in deep photonic neuromorphic networks [10.099714133516608]
  Photonic neuromorphic networks (PNNs) leverage the inherent advantages of light, namely high parallelism, low latency, and exceptional energy efficiency.<n>Here, we introduce a purely photonic deep PNN architecture that enables online, unsupervised learning.<n>We experimentally demonstrate this approach on a non-trivial letter recognition task using a commercially available fiber-optic platform and achieve a 100 percent recognition rate.
  arXiv  Detail & Related papers  (2026-01-29T20:26:36Z)
• Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning [78.88684753303794]
  Deep learning has predominantly advanced through applications in computer vision and natural language processing.<n>Neural operators are a principled way to generalize neural networks to mappings between function spaces.<n>This paper identifies and distills the key principles for constructing practical implementations of mappings between infinite-dimensional function spaces.
  arXiv  Detail & Related papers  (2025-06-12T17:59:31Z)
• Nonlinear Computation with Linear Optics via Source-Position Encoding [0.0]
  We introduce a novel method to achieve nonlinear computation in fully linear media.<n>Our method can operate at low power and requires only the ability to drive the optical system at a data-dependent spatial position.<n>We formulate a fully automated, topology-optimization-based hardware design framework for extremely specialized optical neural networks.
  arXiv  Detail & Related papers  (2025-04-29T03:55:05Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• A Review of Neuroscience-Inspired Machine Learning [58.72729525961739]
  Bio-plausible credit assignment is compatible with practically any learning condition and is energy-efficient. In this paper, we survey several vital algorithms that model bio-plausible rules of credit assignment in artificial neural networks. We conclude by discussing the future challenges that will need to be addressed in order to make such algorithms more useful in practical applications.
  arXiv  Detail & Related papers  (2024-02-16T18:05:09Z)
• ShadowNet for Data-Centric Quantum System Learning [188.683909185536]
  We propose a data-centric learning paradigm combining the strength of neural-network protocols and classical shadows. Capitalizing on the generalization power of neural networks, this paradigm can be trained offline and excel at predicting previously unseen systems. We present the instantiation of our paradigm in quantum state tomography and direct fidelity estimation tasks and conduct numerical analysis up to 60 qubits.
  arXiv  Detail & Related papers  (2023-08-22T09:11:53Z)
• A Survey on Brain-Inspired Deep Learning via Predictive Coding [85.93245078403875]
  Predictive coding (PC) has shown promising performance in machine intelligence tasks.<n>PC can model information processing in different brain areas, can be used in cognitive control and robotics.
  arXiv  Detail & Related papers  (2023-08-15T16:37:16Z)
• Neuro-Symbolic Artificial Intelligence (AI) for Intent based Semantic Communication [85.06664206117088]
  6G networks must consider semantics and effectiveness (at end-user) of the data transmission. NeSy AI is proposed as a pillar for learning causal structure behind the observed data. GFlowNet is leveraged for the first time in a wireless system to learn the probabilistic structure which generates the data.
  arXiv  Detail & Related papers  (2022-05-22T07:11:57Z)
• Physics-informed ConvNet: Learning Physical Field from a Shallow Neural Network [0.180476943513092]
  Modelling and forecasting multi-physical systems remain a challenge due to unavoidable data scarcity and noise. New framework named physics-informed convolutional network (PICN) is recommended from a CNN perspective. PICN may become an alternative neural network solver in physics-informed machine learning.
  arXiv  Detail & Related papers  (2022-01-26T14:35:58Z)
• Credit Assignment in Neural Networks through Deep Feedback Control [59.14935871979047]
  Deep Feedback Control (DFC) is a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing.
  arXiv  Detail & Related papers  (2021-06-15T05:30:17Z)
• A deep learning theory for neural networks grounded in physics [2.132096006921048]
  We argue that building large, fast and efficient neural networks on neuromorphic architectures requires rethinking the algorithms to implement and train them. Our framework applies to a very broad class of models, namely systems whose state or dynamics are described by variational equations.
  arXiv  Detail & Related papers  (2021-03-18T02:12:48Z)
• Reservoir Computing with Magnetic Thin Films [35.32223849309764]
  New unconventional computing hardware has emerged with the potential to exploit natural phenomena and gain efficiency. Physical reservoir computing demonstrates this with a variety of unconventional systems. We perform an initial exploration of three magnetic materials in thin-film geometries via microscale simulation.
  arXiv  Detail & Related papers  (2021-01-29T17:37:17Z)
• Deep neural networks for the evaluation and design of photonic devices [0.0]
  Review: How deep neural networks can learn from training sets and operate as high-speed surrogate electromagnetic solvers. Fundamental data sciences framed within the context of photonics will also be discussed.
  arXiv  Detail & Related papers  (2020-06-30T19:52:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.