The Homogeneity Trap: Spectral Collapse in Doubly-Stochastic Deep Networks

• URL: http://arxiv.org/abs/2601.02080v1
• Date: Mon, 05 Jan 2026 13:09:42 GMT
• Title: The Homogeneity Trap: Spectral Collapse in Doubly-Stochastic Deep Networks
• Authors: Yizhi Liu,
• Abstract summary: We identify a critical spectral degradation phenomenon inherent to structure-preserving deep architectures.<n>We show that maximum-entropy bias drives the mixing operator towards the uniform barycenter, suppressing the subdominant singular value .<n>We derive a spectral bound linking to the network's effective depth, showing that high-entropy constraints restrict feature transformation to a shallow receptive field.
• Score: 1.7523718031184992
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Doubly-stochastic matrices (DSM) are increasingly utilized in structure-preserving deep architectures -- such as Optimal Transport layers and Sinkhorn-based attention -- to enforce numerical stability and probabilistic interpretability. In this work, we identify a critical spectral degradation phenomenon inherent to these constraints, termed the Homogeneity Trap. We demonstrate that the maximum-entropy bias, typical of Sinkhorn-based projections, drives the mixing operator towards the uniform barycenter, thereby suppressing the subdominant singular value σ_2 and filtering out high-frequency feature components. We derive a spectral bound linking σ_2 to the network's effective depth, showing that high-entropy constraints restrict feature transformation to a shallow effective receptive field. Furthermore, we formally demonstrate that Layer Normalization fails to mitigate this collapse in noise-dominated regimes; specifically, when spectral filtering degrades the Signal-to-Noise Ratio (SNR) below a critical threshold, geometric structure is irreversibly lost to noise-induced orthogonal collapse. Our findings highlight a fundamental trade-off between entropic stability and spectral expressivity in DSM-constrained networks.

Related papers

• The Malignant Tail: Spectral Segregation of Label Noise in Over-Parameterized Networks [0.0]
  We experimentally isolate the Malignant Tail, a failure mode where networks functionally segregate signal and noise.<n>We show that untrained networks actively segregate noise, allowing post-hoc Explicit Spectral Truncation to surgically prune the noise-dominated subspace.<n>Our findings suggest that under label noise, excess spectral capacity is not harmless redundancy but a latent structural liability.
  arXiv  Detail & Related papers  (2026-03-02T16:39:42Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Laplacian-LoRA: Delaying Oversmoothing in Deep GCNs via Spectral Low-Rank Adaptation [0.0]
  We propose Laplacian-LoRA, a low-rank adaptation of standard graph convolutional networks (GCNs)<n>Rather than redesigning message passing, Laplacian-LoRA introduces a learnable, spectrally anchored correction to the fixed Laplacian propagation operator.<n>We show that Laplacian-LoRA consistently delays the onset of oversmoothing, extending the effective depth of GCNs by up to a factor of two.
  arXiv  Detail & Related papers  (2026-02-07T00:03:19Z)
• On the Spectral Flattening of Quantized Embeddings [25.64641307046705]
  Training Large Language Models at ultra-low precision is critically impeded by instability rooted in the conflict between discrete quantization constraints and the intrinsic heavy-tailed spectral nature of linguistic data.<n>This work not only quantifies the spectral sensitivity of LLMs but also establishes spectral fidelity as a necessary condition for stable low-bit optimization.
  arXiv  Detail & Related papers  (2026-02-01T02:21:53Z)
• Frequency Regularization: Unveiling the Spectral Inductive Bias of Deep Neural Networks [21.749207209704906]
  We investigate the Spectral Bias of modern Convolutional Neural Networks (CNNs)<n>We introduce a Visual Diagnostic Framework to track the dynamic evolution of weight frequencies during training.<n>We propose a novel metric, the Spectral Suppression Ratio (SSR), to quantify the "low-pass filtering" intensity of different regularizers.
  arXiv  Detail & Related papers  (2025-12-20T11:33:32Z)
• Lazy Diffusion: Mitigating spectral collapse in generative diffusion-based stable autoregressive emulation of turbulent flows [0.0]
  We show that standard DDPMs induce a fundamental emphspectral collapse.<n>We introduce power-law schedules that preserve fine-scale structure deeper into diffusion time.<n>These methods are applied to high-Reynolds-number 2D Kolmogorov turbulence and $1/12circ$ Gulf of Mexico ocean reanalysis.
  arXiv  Detail & Related papers  (2025-12-10T12:05:32Z)
• Spectral Graph Clustering under Differential Privacy: Balancing Privacy, Accuracy, and Efficiency [53.98433419539793]
  We study the problem of spectral graph clustering under edge differential privacy (DP)<n>Specifically, we develop three mechanisms: (i) graph perturbation via randomized edge flipping combined with adjacency matrix shuffling, which enforces edge privacy; (ii) private graph projection with additive Gaussian noise in a lower-dimensional space to reduce dimensionality and computational complexity; and (iii) a noisy power iteration method that distributes Gaussian noise across iterations to ensure edge DP while maintaining convergence.
  arXiv  Detail & Related papers  (2025-10-08T15:30:27Z)
• Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization [49.85875869048434]
  We propose a CP-based low-rank tensor function parameterized by neural networks for implicit neural representation.<n>To achieve sparser CP decomposition, we introduce a variational Schatten-p quasi-norm to prune redundant rank-1 components.<n>For smoothness, we propose a regularization term based on the spectral norm of the Jacobian and Hutchinson's trace estimator.
  arXiv  Detail & Related papers  (2025-06-27T11:23:10Z)
• Controlling Excitation Localization in Waveguide QED Systems [0.4999814847776098]
  We study localization and long-time population trapping in quantum emitters coupled to a waveguide.<n>We find two distinct mechanisms that give rise to localization: geometry-induced subradiance and disorder-induced Anderson-like confinement.<n>These results establish geometry and disorder as complementary tools for engineering long-lived quantum states in waveguide QED systems.
  arXiv  Detail & Related papers  (2025-05-27T08:24:51Z)
• Spectral-Spatial Extraction through Layered Tensor Decomposition for Hyperspectral Anomaly Detection [6.292153194561472]
  Low rank tensor representation (LRTR) methods are very useful for hyperspectral anomaly detection (HAD)<n>We first apply non-negative matrix factorization (NMF) to alleviate spectral dimensionality redundancy and extract spectral anomaly.<n>We then employ LRTR to extract spatial anomaly while mitigating spatial redundancy, yielding a highly efffcient layered tensor decomposition framework for HAD.<n> Experimental results on the Airport-Beach-Urban and MVTec datasets demonstrate that our approach outperforms state-of-the-art methods in the HAD task.
  arXiv  Detail & Related papers  (2025-03-07T07:08:14Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• Noise-resilient Edge Modes on a Chain of Superconducting Qubits [103.93329374521808]
  Inherent symmetry of a quantum system may protect its otherwise fragile states. We implement the one-dimensional kicked Ising model which exhibits non-local Majorana edge modes (MEMs) with $mathbbZ$ parity symmetry. MEMs are found to be resilient against certain symmetry-breaking noise owing to a prethermalization mechanism.
  arXiv  Detail & Related papers  (2022-04-24T22:34:15Z)
• Robust, Nonparametric, Efficient Decomposition of Spectral Peaks under Distortion and Interference [0.0]
  We propose a decomposition method for the spectral peaks in an observed frequency spectrum, which is efficiently acquired by utilizing the Fast Fourier Transform. We model the peaks in spectrum as pseudo-symmetric functions, where the only constraint is a nonincreasing behavior around a central frequency when the distance increases. Our approach is more robust against arbitrary distortion, interference and noise on the spectrum that may be caused by an observation system.
  arXiv  Detail & Related papers  (2022-04-18T17:08:37Z)
• Hyperspectral Image Denoising Using Non-convex Local Low-rank and Sparse Separation with Spatial-Spectral Total Variation Regularization [49.55649406434796]
  We propose a novel non particular approach to robust principal component analysis for HSI denoising. We develop accurate approximations to both rank and sparse components. Experiments on both simulated and real HSIs demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-01-08T11:48:46Z)
• Quantum correlations, entanglement spectrum and coherence of two-particle reduced density matrix in the Extended Hubbard Model [62.997667081978825]
  We study the ground state properties of the one-dimensional extended Hubbard model at half-filling. In particular, in the superconducting region, we obtain that the entanglement spectrum signals a transition between a dominant singlet (SS) to triplet (TS) pairing ordering in the system.
  arXiv  Detail & Related papers  (2021-10-29T21:02:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.