Unlocked Backpropagation using Wave Scattering

• URL: http://arxiv.org/abs/2602.10461v1
• Date: Wed, 11 Feb 2026 03:00:06 GMT
• Title: Unlocked Backpropagation using Wave Scattering
• Authors: Christian Pehle, Jean-Jacques Slotine,
• Abstract summary: We derive a reformulation of the maximum principle in optimal control theory as a hyperbolic initial value problem.<n>This relaxation of the original problem can be interpreted as a physical system that equilibrates.<n>We discretize this continuum theory to derive a family of fully unlocked algorithms suitable for training neural networks.
• Score: 0.006486143522483092
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Both the backpropagation algorithm in machine learning and the maximum principle in optimal control theory are posed as a two-point boundary problem, resulting in a "forward-backward" lock. We derive a reformulation of the maximum principle in optimal control theory as a hyperbolic initial value problem by introducing an additional "optimization time" dimension. We introduce counter-propagating wave variables with finite propagation speed and recast the optimization problem in terms of scattering relationships between them. This relaxation of the original problem can be interpreted as a physical system that equilibrates and changes its physical properties in order to minimize reflections. We discretize this continuum theory to derive a family of fully unlocked algorithms suitable for training neural networks. Different parameter dynamics, including gradient descent, can be derived by demanding dissipation and minimization of reflections at parameter ports. These results also imply that any physical substrate that supports the scattering and dissipation of waves can be interpreted as solving an optimization problem.

Related papers

• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Self-Supervised Coarsening of Unstructured Grid with Automatic Differentiation [55.88862563823878]
  In this work, we present an original algorithm to coarsen an unstructured grid based on the concepts of differentiable physics.<n>We demonstrate performance of the algorithm on two PDEs: a linear equation which governs slightly compressible fluid flow in porous media and the wave equation.<n>Our results show that in the considered scenarios, we reduced the number of grid points up to 10 times while preserving the modeled variable dynamics in the points of interest.
  arXiv  Detail & Related papers  (2025-07-24T11:02:13Z)
• Kernel-Adaptive PI-ELMs for Forward and Inverse Problems in PDEs with Sharp Gradients [0.0]
  This paper introduces the Kernel Adaptive Physics-Informed Extreme Learning Machine (KAPI-ELM)<n>It is designed to solve both forward and inverse Partial Differential Equation (PDE) problems involving localized sharp gradients.<n>KAPI-ELM achieves state-of-the-art accuracy in both forward and inverse settings.
  arXiv  Detail & Related papers  (2025-07-14T13:03:53Z)
• Towards Understanding the Optimization Mechanisms in Deep Learning [5.281849820329249]
  In this paper, we adopt a distribution estimation perspective to explore the mechanisms of supervised classification using deep neural networks.<n>For the latter, we provide theoretical insights into mechanisms such as over- and probability randomization.
  arXiv  Detail & Related papers  (2025-03-29T08:46:13Z)
• Neural Time-Reversed Generalized Riccati Equation [60.92253836775246]
  Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. This paper introduces a novel neural-based approach to optimal control, with the aim of working forward-in-time.
  arXiv  Detail & Related papers  (2023-12-14T19:29:37Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• A Deep Unrolling Model with Hybrid Optimization Structure for Hyperspectral Image Deconvolution [50.13564338607482]
  We propose a novel optimization framework for the hyperspectral deconvolution problem, called DeepMix.<n>It consists of three distinct modules, namely, a data consistency module, a module that enforces the effect of the handcrafted regularizers, and a denoising module.<n>This work proposes a context aware denoising module designed to sustain the advancements achieved by the cooperative efforts of the other modules.
  arXiv  Detail & Related papers  (2023-06-10T08:25:16Z)
• Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks [2.2713084727838115]
  We introduce a novel approach for analyzing the training dynamics of ReLU networks by examining the characteristic activation boundaries of individual neurons. Our proposed analysis reveals a critical instability in common neural network parameterizations and normalizations during convergence optimization, which impedes fast convergence and hurts performance.
  arXiv  Detail & Related papers  (2023-05-25T10:19:13Z)
• Backpropagation of Unrolled Solvers with Folded Optimization [55.04219793298687]
  The integration of constrained optimization models as components in deep networks has led to promising advances on many specialized learning tasks. One typical strategy is algorithm unrolling, which relies on automatic differentiation through the operations of an iterative solver. This paper provides theoretical insights into the backward pass of unrolled optimization, leading to a system for generating efficiently solvable analytical models of backpropagation.
  arXiv  Detail & Related papers  (2023-01-28T01:50:42Z)
• Towards a machine learning pipeline in reduced order modelling for inverse problems: neural networks for boundary parametrization, dimensionality reduction and solution manifold approximation [0.0]
  Inverse problems, especially in a partial differential equation context, require a huge computational load. We apply a numerical pipeline that involves artificial neural networks to parametrize the boundary conditions of the problem in hand. It derives a general framework capable to provide an ad-hoc parametrization of the inlet boundary and quickly converges to the optimal solution.
  arXiv  Detail & Related papers  (2022-10-26T14:53:07Z)
• Manipulating the Dynamics of a Fermi Resonance with Light. A Direct Optimal Control Theory Approach [0.0]
  Direct optimal control theory for quantum dynamical problems presents itself as an interesting alternative to the traditional indirect optimal control. We extend the application of the method to the case of exact wavepacket propagation using the example of a generic Fermi-resonance model.
  arXiv  Detail & Related papers  (2021-08-27T14:30:03Z)
• Direct Optimal Control Approach to Laser-Driven Quantum Particle Dynamics [77.34726150561087]
  We propose direct optimal control as a robust and flexible alternative to indirect control theory. The method is illustrated for the case of laser-driven wavepacket dynamics in a bistable potential.
  arXiv  Detail & Related papers  (2020-10-08T07:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.