The role of class encoding in neural collapse

• URL: http://arxiv.org/abs/2606.00344v1
• Date: Fri, 29 May 2026 20:34:07 GMT
• Title: The role of class encoding in neural collapse
• Abstract summary: We explore the role of label encoding in neural collapse by relying on the unrestricted feature model with mean squared error training loss.<n>We show that for one-hot encoded labels and balanced data, the uncentered mean features associated with each class coefficient transition from a simplexangular tight frame to an equixangular tight frame.<n>For any arbitrary encoding, we also show that the final classifier's bias aims at centering the labels, compensating for the discrepancy between the global mean of the labels and the origin.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural collapse is a structural property of the last-hidden-layer activations in neural network classification models, when trained beyond a zero classification error. In this work, we explore the role of label encoding in neural collapse by relying on the unrestricted feature model with mean squared error training loss. We demonstrate that, for one-hot encoded labels and balanced data, the uncentered mean features associated with each class transition from a simplex equiangular tight frame to an orthogonal frame when increasing the bias regularization coefficient associated with the final classifier. These structures are reminiscent of the orthogonal frame structure of one-hot encoded labels. For any arbitrary encoding, we also show that the final classifier's bias aims at centering the labels, compensating for the discrepancy between the global mean of the labels and the origin. We further discuss the role of the encoding in other neural collapse properties.

Related papers

• Benign Overfitting and the Geometry of the Ridge Regression Solution in Binary Classification [75.01389991485098]
  We show that ridge regression has qualitatively different behavior depending on the scale of the cluster mean vector.<n>In regimes where the scale is very large, the conditions that allow for benign overfitting turn out to be the same as those for the regression task.
  arXiv  Detail & Related papers  (2025-03-11T01:45:42Z)
• Open-World Semi-Supervised Learning for Node Classification [53.07866559269709]
  Open-world semi-supervised learning (Open-world SSL) for node classification is a practical but under-explored problem in the graph community. We propose an IMbalance-Aware method named OpenIMA for Open-world semi-supervised node classification.
  arXiv  Detail & Related papers  (2024-03-18T05:12:54Z)
• Neural Collapse in Multi-label Learning with Pick-all-label Loss [28.46350033490066]
  We study deep neural networks for the multi-label classification (MLab) task through the lens of neural collapse (NC) We prove for the first time that a generalized NC phenomenon holds the "pick-all-label" formulation, which we term as MLab.
  arXiv  Detail & Related papers  (2023-10-24T15:07:16Z)
• Semi-Supervised Laplace Learning on Stiefel Manifolds [48.3427853588646]
  We develop the framework Sequential Subspace for graph-based, supervised samples at low-label rates. We achieves that our methods at extremely low rates, and high label rates.
  arXiv  Detail & Related papers  (2023-07-31T20:19:36Z)
• Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity [0.0]
  We propose to fix the linear classifier of a deep neural network to a Hierarchy-Aware Frame (HAFrame) We demonstrate that our approach reduces the mistake severity of the model's predictions while maintaining its top-1 accuracy on several datasets.
  arXiv  Detail & Related papers  (2023-03-10T03:44:01Z)
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning [120.53458753007851]
  Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. We deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse. We propose a neural collapse inspired framework for FSCIL. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances.
  arXiv  Detail & Related papers  (2023-02-06T18:39:40Z)
• Understanding Imbalanced Semantic Segmentation Through Neural Collapse [81.89121711426951]
  We show that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes. We introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure. Our method ranks 1st and sets a new record on the ScanNet200 test leaderboard.
  arXiv  Detail & Related papers  (2023-01-03T13:51:51Z)
• Imbalance Trouble: Revisiting Neural-Collapse Geometry [27.21274327569783]
  We introduce Simplex-Encoded-Labels Interpolation (SELI) as an invariant characterization of the neural collapse phenomenon. We prove for the UFM with cross-entropy loss and vanishing regularization. We present experiments on synthetic and real datasets that confirm convergence to the SELI geometry.
  arXiv  Detail & Related papers  (2022-08-10T18:10:59Z)
• Do We Really Need a Learnable Classifier at the End of Deep Neural Network? [118.18554882199676]
  We study the potential of learning a neural network for classification with the classifier randomly as an ETF and fixed during training. Our experimental results show that our method is able to achieve similar performances on image classification for balanced datasets.
  arXiv  Detail & Related papers  (2022-03-17T04:34:28Z)
• Latent Embedding Feedback and Discriminative Features for Zero-Shot Classification [139.44681304276]
  zero-shot learning aims to classify unseen categories for which no data is available during training. Generative Adrial Networks synthesize unseen class features by leveraging class-specific semantic embeddings. We propose to enforce semantic consistency at all stages of zero-shot learning: training, feature synthesis and classification.
  arXiv  Detail & Related papers  (2020-03-17T17:34:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.