Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning

• URL: http://arxiv.org/abs/2605.11904v1
• Date: Tue, 12 May 2026 10:18:08 GMT
• Title: Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning
• Abstract summary: Nearest Class Mean (NCM) is widely favored in Class-Incremental Learning (CIL) for its superior resistance to catastrophic forgetting.<n>We propose Hierarchical-Cluster SOINN (HC-SOINN), a novel classifier that captures the topological structure of these classes via a local-to-global'' representation.<n>We introduce Structure-Topology Alignment via Residuals (STAR) method, which employs a fine-grained pointwise trajectory tracking mechanism.
• Score: 11.791841292540377
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Nearest Class Mean (NCM) classifier is widely favored in Class-Incremental Learning (CIL) for its superior resistance to catastrophic forgetting compared to Fully Connected layers. While Neural Collapse (NC) theory supports NCM's optimality by assuming features collapse into single points, non-linear feature drift and insufficient training in CIL often prevent this ideal state. Consequently, classes manifest as complex manifolds rather than collapsed points, rendering the single-point NCM suboptimal. To address this, we propose Hierarchical-Cluster SOINN (HC-SOINN), a novel classifier that captures the topological structure of these manifolds via a ``local-to-global'' representation. Furthermore, we introduce Structure-Topology Alignment via Residuals (STAR) method, which employs a fine-grained pointwise trajectory tracking mechanism to actively deform the learned topology, allowing it to adapt precisely to complex non-linear feature drift. Theoretical analysis and Procrustes distance experiments validate our framework's resilience to manifold deformations. We integrated HC-SOINN into seven state-of-the-art methods by replacing their original classifiers, achieving consistent improvements that highlight the effectiveness and robustness of our approach. Code is available at https://github.com/yhyet/HC_SOINN.

Related papers

• Learning to Tessellate: Point Cloud Generation via Recursive Spectral Partitioning [58.130498928480456]
  We present PointRSP, an autoregressive framework that reformulates point cloud generation as a topology-preserving tessellation process.<n>PointRSP achieves state-of-the-art performance in generation quality and diversity, demonstrating strong generalization across complex 3D topologies.
  arXiv  Detail & Related papers  (2026-08-03T16:11:43Z)
• Online Continual Learning with Dynamic Label Hierarchies [72.33335166136045]
  We introduce a new problem setting, DHOCL, where severity evolve across granularities and each sample provides supervision at a single hierarchical level.<n>In this setting, we find two fundamental issues: (i) partial supervision under mixed granularities provides only point-wise signals over an evolving path-wise hierarchy, which constrains plasticity and undermines cross-level semantic consistency.<n>We propose HALO, which adaptively combines complementary classification heads, regularized by organized learnable hierarchical prototypes, enabling rapid adaptation, hierarchical consistency, and structured knowledge consolidation as the taxonomy evolves.
  arXiv  Detail & Related papers  (2026-05-12T08:20:23Z)
• Hyperbolic Enhanced Representation Learning for Incomplete Multi-view Clustering [57.38215918201251]
  We propose HERL, a Hyperbolic Enhanced Representation Learning framework for Incomplete Multi-View Clustering.<n> operating within the Poincaré ball, HERL constructs a structure-aware latent space to enhance representation learning.<n>We show that HERL consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-04-18T10:50:46Z)
• Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints [3.421126849736858]
  We introduce NucGD, a geometry-aware geometry designed to enforce low rank structures through nuclear norm constraints.<n>To enable scalable training, we derive an efficient SVD-free update rule via asynchronous power iteration.<n>We empirically dissect the impact of gradient optimization dynamics, characterizing how varying levels of gradient noise induced by mini-batch sampling and momentum modulate the convergence toward the expected maximum margin solutions.
  arXiv  Detail & Related papers  (2026-03-24T05:51:50Z)
• This Looks Distinctly Like That: Grounding Interpretable Recognition in Stiefel Geometry against Neural Collapse [28.864167134877885]
  Prototype networks provide an intrinsic case based explanation mechanism, but their interpretability is often undermined by prototype collapse.<n>We propose Adaptive Manifold Prototypes (AMP) to represent class prototypes as orthonormal bases and make rank one prototype collapse infeasible by construction.<n>AMP achieves state-of-the-art classification accuracy while significantly improving causal faithfulness over prior interpretable models.
  arXiv  Detail & Related papers  (2026-03-09T13:36:54Z)
• Feedback Alignment Meets Low-Rank Manifolds: A Structured Recipe for Local Learning [7.034739490820967]
  Training deep neural networks (DNNs) with backpropagation (BP) achieves state-of-the-art accuracy but requires global error propagation and full parameterization.<n>Direct Feedback Alignment (DFA) enables local, parallelizable updates with lower memory requirements.<n>We propose a structured local learning framework that operates directly on low-rank manifold.
  arXiv  Detail & Related papers  (2025-10-29T15:03:46Z)
• Deep Hierarchical Learning with Nested Subspace Networks [53.71337604556311]
  We propose Nested Subspace Networks (NSNs) for large neural networks.<n>NSNs enable a single model to be dynamically and granularly adjusted across a continuous spectrum of compute budgets.<n>We show that NSNs can be surgically applied to pre-trained LLMs and unlock a smooth and predictable compute-performance frontier.
  arXiv  Detail & Related papers  (2025-09-22T15:13:14Z)
• Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model [5.339955242953934]
  We show that a phenomenon we call Ordinal Neural Collapse (ONC) indeed emerges and is characterized by the following three properties.<n>In particular, in the zero-regularization limit, a highly local and simple geometric relationship emerges between the latent variables and the threshold values.
  arXiv  Detail & Related papers  (2025-06-06T06:57:02Z)
• Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame [34.309687104447114]
  Neural Collapse (NC) is a recently observed phenomenon in neural networks that characterises the solution space of the final classifier layer when trained until zero training loss. We introduce the notion of nearest simplex ETF geometry for the penultimate layer features at any given training iteration. At each iteration, the classifier weights are implicitly set to the nearest simplex ETF by solving this inner-optimisation. Our experiments on synthetic and real-world architectures for classification tasks demonstrate that our approach accelerates convergence and enhances training stability.
  arXiv  Detail & Related papers  (2024-11-02T13:54:31Z)
• A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks [44.31777384413466]
  Graph neural networks (GNNs) have become increasingly popular for classification tasks on graph-structured data. In this paper, we focus on node-wise classification and explore the feature evolution through the lens of the "Neural Collapse" phenomenon. We show that even an "optimistic" mathematical model requires that the graphs obey a strict structural condition in order to possess a minimizer with exact collapse.
  arXiv  Detail & Related papers  (2023-07-04T23:03:21Z)
• Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning [120.53458753007851]
  Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. We deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse. We propose a neural collapse inspired framework for FSCIL. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances.
  arXiv  Detail & Related papers  (2023-02-06T18:39:40Z)
• On Feature Learning in Neural Networks with Global Convergence Guarantees [49.870593940818715]
  We study the optimization of wide neural networks (NNs) via gradient flow (GF) We show that when the input dimension is no less than the size of the training set, the training loss converges to zero at a linear rate under GF. We also show empirically that, unlike in the Neural Tangent Kernel (NTK) regime, our multi-layer model exhibits feature learning and can achieve better generalization performance than its NTK counterpart.
  arXiv  Detail & Related papers  (2022-04-22T15:56:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.