The role of parameter Jacobians in the stability of network outputs

• URL: http://arxiv.org/abs/2608.27748v1
• Date: Thu, 27 Aug 2026 22:21:10 GMT
• Title: The role of parameter Jacobians in the stability of network outputs
• Abstract summary: We show that the corresponding linearized dynamics leads naturally to a semigroup formulation.<n>We prove norm-bounds on the corresponding semigroup perturbations.<n>We extend the comparison to nonautonomous NTK evolutions through piecewise-frozen approximations.
• Score: 2.000834478067184
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the framework of network dynamics, learning models, and neural tangent kernels (NTK), we show that the corresponding linearized dynamics leads naturally to a semigroup formulation. More precisely, in our analysis of input/output models, the time-dynamics is presented via special semigroups of linear operators on Hilbert spaces, together with an associated class of semigroup perturbations. In this context, we then present new and explicit a priori perturbation-bound results: for the fixed-kernel linearization constructions arising in the NTK setting, we prove norm-bounds on the corresponding semigroup perturbations, in the form of explicit finite-time perturbation estimates. We further present refinements on prescribed task spaces, Cesàro-averaged (ergodic) comparisons estimates, and versions in which the lower spectral edge assumption is replaced by a spectral-distribution condition. We also extend the comparison to nonautonomous NTK evolutions through piecewise-frozen approximations, record a corresponding discrete Euler specialization, and offer worked examples in order to illustrate our perturbation-bound estimates.

Related papers

• Shorting Dynamics and Structured Kernel Regularization [0.0]
  The induced sequence of positive operators is monotone, admits an exact residual decomposition, and converges to the classical shorted operator.<n>This gives a unified operator-analytic approach to invariant kernel construction and structured regularization in data analysis.
  arXiv  Detail & Related papers  (2025-12-04T15:02:42Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks [7.486235601021366]
  Lipschitz continuity characterizes the worst-case sensitivity of neural networks to small input perturbations.<n>We present a rigorous framework to model the temporal evolution of Lipschitz continuity during training with gradient descent.
  arXiv  Detail & Related papers  (2025-06-23T12:49:13Z)
• Scattering Networks on Noncommutative Finite Groups [0.0]
  We introduce a scattering transform on an arbitrary finite group (not necessarily abelian) within the context of group-equivariant convolutional neural networks (G-CNNs)<n>We demonstrate that, under certain conditions in the wavelet coefficients, the scattering transform is non-expansive, stable under deformations, preserves energy, equivariant with respect to left and right group translations, and, as depth increases, the scattering coefficients are less sensitive to group translations of the signal.
  arXiv  Detail & Related papers  (2025-05-27T09:41:39Z)
• Learning convolution operators on compact Abelian groups [35.70694383605516]
  We consider the problem of learning convolution operators associated to compact Abelian groups.<n>We study a regularization-based approach and provide learning guarantees under natural regularity conditions.
  arXiv  Detail & Related papers  (2025-01-09T14:43:08Z)
• Relative Representations: Topological and Geometric Perspectives [50.85040046976025]
  Relative representations are an established approach to zero-shot model stitching.<n>We introduce a normalization procedure in the relative transformation, resulting in invariance to non-isotropic rescalings and permutations.<n>Second, we propose to deploy topological densification when fine-tuning relative representations, a topological regularization loss encouraging clustering within classes.
  arXiv  Detail & Related papers  (2024-09-17T08:09:22Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Feature-aligned N-BEATS with Sinkhorn divergence [8.209488690262882]
  We propose Feature-aligned N-BEATS as a domain-generalized time series forecasting model. It is a nontrivial extension of N-BEATS with doubly residual stacking principle into a representation learning framework.
  arXiv  Detail & Related papers  (2023-05-24T14:32:23Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.