Upper Generalization Bounds for Neural Oscillators

• URL: http://arxiv.org/abs/2603.09742v1
• Date: Tue, 10 Mar 2026 14:47:23 GMT
• Title: Upper Generalization Bounds for Neural Oscillators
• Abstract summary: We study a neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP)<n>Its upper probably approximately correct (PAC) bound for approximating causal and uniformly continuous operators between continuous temporal function spaces.<n>A numerical study considering a Bouc-Wen nonlinear excitation system under seismic conditions validates the theoretically predicted power laws of the estimation errors.
• Score: 8.075776288865907
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural oscillators that originate from the second-order ordinary differential equations (ODEs) have shown competitive performance in learning mappings between dynamic loads and responses of complex nonlinear structural systems. Despite this empirical success, theoretically quantifying the generalization capacities of their neural network architectures remains undeveloped. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper probably approximately correct (PAC) generalization bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating the uniformly asymptotically incrementally stable second-order dynamical systems are derived by leveraging the Rademacher complexity framework. The theoretical results show that the estimation errors grow polynomially with respect to both the MLP size and the time length, thereby avoiding the curse of parametric complexity. Furthermore, the derived error bounds demonstrate that constraining the Lipschitz constants of the MLPs via loss function regularization can improve the generalization ability of the neural oscillator. A numerical study considering a Bouc-Wen nonlinear system under stochastic seismic excitation validates the theoretically predicted power laws of the estimation errors with respect to the sample size and time length, and confirms the effectiveness of constraining MLPs' matrix and vector norms in enhancing the performance of the neural oscillator under limited training data.

Related papers

• A Measure-Theoretic Finite-Sample Theory for Adaptive-Data Fitted Q-Iteration [14.808143352500531]
  We develop a framework for fitted Q-iteration (FQI) on general measurable Borel spaces.<n>We prove that sequential Rademacher complexity controls Bellman-regression generalization under policy-dependent data collection.<n>These results lay the necessary foundations for the formal analysis of many modern deep RL algorithms.
  arXiv  Detail & Related papers  (2026-05-07T07:26:58Z)
• KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra [65.11254608352982]
  We introduce a generator-based neural Koopman framework that models dynamics through a structured, state-dependent representation of Koopman generators.<n>By exploiting the intrinsic Cartesian decomposition into skew-adjoint and self-adjoint components, KoopGen separates conservative transport from irreversible dissipation.
  arXiv  Detail & Related papers  (2026-02-15T06:32:23Z)
• Upper Approximation Bounds for Neural Oscillators [8.075776288865907]
  Theory of quantifying the capacities of neural network architectures remains a significant challenge.<n>This study considers the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron.<n>Results provide a robust theoretical foundation for the effective application of the neural oscillator in science and engineering.
  arXiv  Detail & Related papers  (2025-11-30T18:20:40Z)
• Delay-adaptive Control of Nonlinear Systems with Approximate Neural Operator Predictors [6.093618731228799]
  We propose a rigorous method for implementing predictor feedback controllers in nonlinear systems with unknown and arbitrarily long actuator delays.<n>To address the analytically intractable nature of the predictor, we approximate it using a learned neural operator mapping.<n>We provide a theoretical stability analysis based on the universal approximation theorem of neural operators and the transport partial differential equation (PDE) representation of the delay.<n>We then prove, via a Lyapunov-Krasovskii functional, semi-global practical convergence of the dynamical system dependent on the approximation error of the predictor and delay bounds.
  arXiv  Detail & Related papers  (2025-08-28T02:30:53Z)
• Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks [7.486235601021366]
  Lipschitz continuity characterizes the worst-case sensitivity of neural networks to small input perturbations.<n>We present a rigorous framework to model the temporal evolution of Lipschitz continuity during training with gradient descent.
  arXiv  Detail & Related papers  (2025-06-23T12:49:13Z)
• On the Generalization and Approximation Capacities of Neural Controlled Differential Equations [0.3222802562733786]
  Neural Controlled Differential Equations (NCDEs) are a state-of-the-art tool for supervised learning with irregularly sampled time series. We show how classical approximation results on neural nets may transfer to NCDEs.
  arXiv  Detail & Related papers  (2023-05-26T10:02:32Z)
• Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iteration [122.51142131506639]
  We introduce a precise, fast, and differentiable upper bound for the spectral norm of convolutional layers using circulant matrix theory. We show through a comprehensive set of experiments that our approach outperforms other state-of-the-art methods in terms of precision, computational cost, and scalability. It proves highly effective for the Lipschitz regularization of convolutional neural networks, with competitive results against concurrent approaches.
  arXiv  Detail & Related papers  (2023-05-25T15:32:21Z)
• Sharp Spectral Rates for Koopman Operator Learning [27.820383937933034]
  We present for the first time non-asymptotic learning bounds for the Koopman eigenvalues and eigenfunctions. Our results shed new light on the emergence of spurious eigenvalues.
  arXiv  Detail & Related papers  (2023-02-03T21:19:56Z)
• Temporal Difference Learning with Compressed Updates: Error-Feedback meets Reinforcement Learning [47.904127007515925]
  We study a variant of the classical temporal difference (TD) learning algorithm with a perturbed update direction. We prove that compressed TD algorithms, coupled with an error-feedback mechanism used widely in optimization, exhibit the same non-asymptotic approximation guarantees as their counterparts. Notably, these are the first finite-time results in RL that account for general compression operators and error-feedback in tandem with linear function approximation and Markovian sampling.
  arXiv  Detail & Related papers  (2023-01-03T04:09:38Z)
• Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms [71.62575565990502]
  We prove that the generalization error of an optimization algorithm can be bounded on the complexity' of the fractal structure that underlies its generalization measure. We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden/layered neural networks) and algorithms.
  arXiv  Detail & Related papers  (2021-06-09T08:05:36Z)
• Consistency of mechanistic causal discovery in continuous-time using Neural ODEs [85.7910042199734]
  We consider causal discovery in continuous-time for the study of dynamical systems. We propose a causal discovery algorithm based on penalized Neural ODEs.
  arXiv  Detail & Related papers  (2021-05-06T08:48:02Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
  empirical optimization is central to modern machine learning, but its role in its success is still unclear. We show that it commonly arises in parameters of discrete multiplicative noise due to variance. A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
  arXiv  Detail & Related papers  (2020-06-11T09:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.