A Commutator Framework for Selective Spectral Alignment in Deep Neural Networks

• URL: http://arxiv.org/abs/2608.22910v1
• Date: Mon, 24 Aug 2026 07:42:16 GMT
• Title: A Commutator Framework for Selective Spectral Alignment in Deep Neural Networks
• Abstract summary: We develop a finite-width geometric framework describing how learned feature geometries are organized, transported, and selectively aligned in deep neural networks.<n>Incompatibility among weight-generated covariance, gates, and backward sensitivities is quantified through three families of commutators.<n>We establish spectral-gap, projector-evolution, and stabilization estimates, and formulate conditional Lyapunov principles that yield decay under explicit geometric error-bound or intrinsic-damping assumptions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop a finite-width geometric framework describing how learned feature geometries are organized, transported, and selectively aligned in deep neural networks. Incompatibility among weight-generated covariance, gates, and backward sensitivities is quantified through three families of commutators: between gates and covariance, between sensitivities and covariance, and between average gradient outer products (AGOPs) and neural feature matrices (NFMs). An exact layerwise identity decomposes the sensitivity-covariance commutator into four sources: downstream transport, adjacent-layer imbalance, pointwise sensitivity fluctuations, and nonlinear gate-covariance interactions. The AGOP-NFM commutator is a singular-value-weighted transport of the internal commutator, explaining why observed feature-side alignment alone does not determine the internal geometry from which it emerges. Buffered localized energies resolve mixing between separated covariance subspaces. We establish spectral-gap, projector-evolution, and stabilization estimates, and formulate conditional Lyapunov principles that yield decay under explicit geometric error-bound or intrinsic-damping assumptions. These criteria do not follow from gradient flow alone and clarify why risk reduction need not imply commutator collapse. Analytic examples and numerical experiments exhibit factorization of spectral and activation geometry, transient growth, and cancellation among nonzero sources. In tested finite-time regimes, cancellation dominated by a negative transport-imbalance interaction persists across depths, widths, and two regression benchmarks. Spectral alignment therefore appears as a layer- and scale-dependent compatibility phenomenon governed by transport, interaction, cancellation, and possible damping, rather than a universal consequence of training.

Related papers

• Nonlocality-induced critical-length hierarchy from non-Hermitian competition [48.9823527611878]
  In short-range systems formed by two coupled chains, competition leads to the critical-length law $N_csimln D$.<n>Here we show that long-range hoppings fundamentally reorganizes this critical behavior, producing a hierarchy of distinct scaling laws.<n>Our results offer a platform-independent framework testable in programmable topoelectrical circuits, photonic lattices and digital quantum simulators.
  arXiv  Detail & Related papers  (2026-08-03T18:00:18Z)
• Learning Bidirectional Causal Interactions with Heteroscedastic Neural Networks [1.0356094515229846]
  This paper proposes SEM-DNN, a heteroscedastic neural simultaneous-equation estimator.<n>It learns reciprocal structural interactions without external instruments.<n>An application to ready-to-eat cereal scanner data illustrates how the method can study contemporaneous price-sales feedback.
  arXiv  Detail & Related papers  (2026-07-24T13:53:58Z)
• Bypassing Minimization Bias: A Shift-Invariant Variance Estimator for Off-Equilibrium Local Learning Coefficients [0.0]
  Shift-Invariant Variance Estimator (SIVE)<n>SIVE is a variance-based local LLC probe that structurally eliminates the unknown additive baseline through the variance operator.<n>Applying to deep neural networks, SIVE provides a robust, localized online diagnostic for tracking structural phase transitions throughout training.
  arXiv  Detail & Related papers  (2026-06-21T08:32:34Z)
• A Gauge-Covariant Theoretical Framework for Non-Abelian Holonomy Estimation and Feed-Forward Correction in Time-Bin Photonic Qudits [0.0]
  We develop a theoretical framework for estimating and correcting non-Abelian geometric distortions in time-bin photonic qudit processing.<n>The work does not assume a device-specific transfer matrix, loss model, detector model, or experimental calibration pipeline.
  arXiv  Detail & Related papers  (2026-05-26T08:40:26Z)
• Commutator-Induced Uncertainty in VAEs [20.84327211490012]
  Variational autoencoders (VAEs) often struggle to represent non-commutative structure in learned latent spaces.<n>We introduce a Lie Group VAE framework that combines geometric and algebraic perspectives on uncertainty.<n>We evaluate the framework on dSprites, 3DShapes, 3DCars, and CelebA against generic and symmetry-aware baselines.
  arXiv  Detail & Related papers  (2026-05-22T10:03:46Z)
• Gauge-Equivariant Intrinsic Neural Operators for Geometry-Consistent Learning of Elliptic PDE Maps [0.0]
  We propose Gauge-Equivariant Intrinsic Neural Operators (GINO) for geometric partial differential equations (PDEs)<n>GINO parameterize elliptic solution maps through intrinsic spectral multipliers acting on geometry-dependent spectra, coupled with gauge-equivariant nonlinearities.<n>We show GINO achieves low operator-approximation error, near machine-precision gauge equivariance, equi robustness to structured metric perturbations, strong cross-resolution generalization with small commutation error under restriction/prolongation, and structure-preserving performance on a regularized exact/coexact decomposition task.
  arXiv  Detail & Related papers  (2026-03-16T02:10:06Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Non Commutative Convolutional Signal Models in Neural Networks: Stability to Small Deformations [111.27636893711055]
  We study the filtering and stability properties of non commutative convolutional filters. Our results have direct implications for group neural networks, multigraph neural networks and quaternion neural networks.
  arXiv  Detail & Related papers  (2023-10-05T20:27:22Z)
• Learning Discretized Neural Networks under Ricci Flow [48.47315844022283]
  We study Discretized Neural Networks (DNNs) composed of low-precision weights and activations.<n>DNNs suffer from either infinite or zero gradients due to the non-differentiable discrete function during training.
  arXiv  Detail & Related papers  (2023-02-07T10:51:53Z)
• On Convergence of Training Loss Without Reaching Stationary Points [62.41370821014218]
  We show that Neural Network weight variables do not converge to stationary points where the gradient the loss function vanishes. We propose a new perspective based on ergodic theory dynamical systems.
  arXiv  Detail & Related papers  (2021-10-12T18:12:23Z)
• Convolutional Filtering and Neural Networks with Non Commutative Algebras [153.20329791008095]
  We study the generalization of non commutative convolutional neural networks. We show that non commutative convolutional architectures can be stable to deformations on the space of operators.
  arXiv  Detail & Related papers  (2021-08-23T04:22:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.