How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse

• URL: http://arxiv.org/abs/2605.01897v1
• Date: Sun, 03 May 2026 14:27:12 GMT
• Title: How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse
• Abstract summary: This work investigates the phenomenon of Neural Collapse (NC) in multi-label classification.<n>It extends its conceptual framework from multi-class learning to general correlated and imbalanced multi-label settings.
• Score: 8.294105732989783
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work investigates the phenomenon of Neural Collapse (NC) in multi-label classification, extending its conceptual framework from multi-class learning to general correlated and imbalanced multi-label settings. Although recent studies have identified a ''tag-wise averaging'' structure for multi-label features, this view relies on implicit assumptions of label balance and combinatorial symmetry. Consequently, it fails to account for the geometrical distortions caused by intrinsic label correlations and data imbalance, which are common in practice. We resolve the multiplicity-one imbalance conjecture raised by Li et al. (2024), showing that higher-multiplicity prototypes obey a class-frequency-weighted synthesis rule rather than uniform averaging. To address this, we propose a rigorous spectral-control framework to analyze the terminal phase of multi-label learning under general imbalanced conditions. We introduce the label covariance spectrum $κ_m$, a scalar controlling the distribution-dependent lower-bound geometry, derived from the second-order moment matrix of the label distribution. Contrary to the averaging perspective, our analysis reveals that the centered label covariance spectrum controls the stability of terminal geometry by quantifying the weakest centered inter-class contrast directions. We prove that the classical Tag-wise Averaging emerges only as a special case under perfect orthogonality. Numerical experiments on synthetic distributions validate our theoretical bounds. This work resolves the scaled-average aspect of the imbalance conjecture and establishes a unifying theoretical framework that extends Neural Collapse to complex, imbalanced multi-label settings.

Related papers

• A Spectral Theory of Normalized Corrected GNN Propagation [2.9801823421431854]
  We develop a spectral theory for emphnormalized corrected GNN propagation.<n>The central theoretical question is whether this corrected normalized operator preserves class-discnative signal after many propagation layers.<n>We also establish a multi-class partial recovery theorem showing contraction toward class centers for most nodes.
  arXiv  Detail & Related papers  (2026-06-22T16:38:50Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Spectral Decimation of Quantum Many-Body Hamiltonians [1.1327508153380308]
  We develop a theory of spectral decimation for quantum many-body Hamiltonians.<n>We show that it provides a quantitative probe of emergent symmetries in statistically mixed spectra.
  arXiv  Detail & Related papers  (2026-02-23T19:00:05Z)
• Representation Geometry as a Diagnostic for Out-of-Distribution Robustness [0.5978543974412219]
  We propose a geometry-based diagnostic framework that constructs class-conditional mutual k-nearest-neighbor graphs from in-distribution embeddings.<n>We find that lower spectral complexity and higher mean curvature consistently predict stronger out-of-distribution (OOD) accuracy across checkpoints.<n>Our results demonstrate that representation geometry enables interpretable, label-free diagnosis and supports reliable unsupervised checkpoint selection.
  arXiv  Detail & Related papers  (2026-02-03T19:13:36Z)
• Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits [72.0643009153473]
  We show that expressive variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size.<n>As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions.<n>We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class.
  arXiv  Detail & Related papers  (2026-01-05T08:04:33Z)
• Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance [11.924440950433658]
  We introduce the concept of spectral imbalance in features as a potential source for class disparities. We derive exact expressions for the per-class error in a high-dimensional mixture model setting. We study this phenomenon in 11 different state-of-the-art pretrained encoders.
  arXiv  Detail & Related papers  (2024-02-18T23:59:54Z)
• Imbalance Trouble: Revisiting Neural-Collapse Geometry [27.21274327569783]
  We introduce Simplex-Encoded-Labels Interpolation (SELI) as an invariant characterization of the neural collapse phenomenon. We prove for the UFM with cross-entropy loss and vanishing regularization. We present experiments on synthetic and real datasets that confirm convergence to the SELI geometry.
  arXiv  Detail & Related papers  (2022-08-10T18:10:59Z)
• Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning [80.05441565830726]
  This paper addresses imbalanced semi-supervised learning, where heavily biased pseudo-labels can harm the model performance. We propose a general pseudo-labeling framework to address the bias motivated by this observation. We term the novel pseudo-labeling framework for imbalanced SSL as Distribution-Aware Semantics-Oriented (DASO) Pseudo-label.
  arXiv  Detail & Related papers  (2021-06-10T11:58:25Z)
• Spectral clustering under degree heterogeneity: a case for the random walk Laplacian [83.79286663107845]
  This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. In the special case of a degree-corrected block model, the embedding concentrates about K distinct points, representing communities.
  arXiv  Detail & Related papers  (2021-05-03T16:36:27Z)
• GRACE: Gradient Harmonized and Cascaded Labeling for Aspect-based Sentiment Analysis [90.43089622630258]
  We propose a GRadient hArmonized and CascadEd labeling model (GRACE) to solve these problems. The proposed model achieves consistency improvement on multiple benchmark datasets and generates state-of-the-art results.
  arXiv  Detail & Related papers  (2020-09-22T13:55:34Z)
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning [126.31716228319902]
  We develop Distribution Aligning Refinery of Pseudo-label (DARP) algorithm. We show that DARP is provably and efficiently compatible with state-of-the-art SSL schemes.
  arXiv  Detail & Related papers  (2020-07-17T09:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.