Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation

• URL: http://arxiv.org/abs/2606.07474v1
• Date: Fri, 05 Jun 2026 17:28:19 GMT
• Title: Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation
• Abstract summary: Unsupervised Continual Learning aims to enable neural networks to learn sequential tasks without labels or access to past data.<n>A major challenge in this setting is Catastrophic Forgetting, where models forget previously learned tasks upon learning new ones.<n>We introduce Unsupervised Continual Clustering (UCC) and propose Forward-Backward Knowledge Distillation for Continual Clustering (FBCC)
• Score: 16.561832598191327
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised Continual Learning (UCL) aims to enable neural networks to learn sequential tasks without labels or access to past data. A major challenge in this setting is Catastrophic Forgetting, where models forget previously learned tasks upon learning new ones. This challenge is amplified in UCL due to the absence of labels to guide learning and memory retention. Existing mitigation strategies, such as knowledge distillation and replay buffers, often raise memory and privacy concerns. Moreover, current UCL methods largely overlook clustering-specific objectives. To fill this gap, we introduce Unsupervised Continual Clustering (UCC) and propose Forward-Backward Knowledge Distillation for Continual Clustering (FBCC). FBCC employs a continual teacher network with a clustering projector and lightweight task-specific students. Through a dual-phase forward-backward distillation process, the teacher learns new clusters while preserving previously discovered cluster structure without storing past data. FBCC represents a pioneering approach to UCC, demonstrating improved clustering performance across sequential tasks. Experiments on four benchmark datasets demonstrate that FBCC consistently outperforms existing continual learning baselines in clustering accuracy while significantly reducing catastrophic forgetting.

Related papers

• Group and Exclusive Sparse Regularization-based Continual Learning of CNNs [0.2897702129074362]
  Group and Exclusive Sparsity based Continual Learning avoids forgetting of previous tasks.<n>GESCL makes the network plastic via a plasticity regularization term.<n> Experiments on popular CL vision benchmarks show that GESCL leads to significant improvements over state-of-the-art method.
  arXiv  Detail & Related papers  (2026-01-07T07:15:11Z)
• Clustering via Self-Supervised Diffusion [6.9158153233702935]
  Clustering via Diffusion (CLUDI) is a self-supervised framework that combines the generative power of diffusion models with pre-trained Vision Transformer features to achieve robust and accurate clustering.<n>CLUDI is trained via a teacher-student paradigm: the teacher uses diffusion-based sampling to produce diverse cluster assignments, which the student refines into stable predictions.
  arXiv  Detail & Related papers  (2025-07-06T07:57:08Z)
• Fuzzy Cluster-Aware Contrastive Clustering for Time Series [1.435214708535728]
  Traditional unsupervised clustering methods often fail to capture the complex nature of time series data.<n>We propose a fuzzy cluster-aware contrastive clustering framework (FCACC) that jointly optimize representation learning and clustering.<n>Our approach introduces a novel three-view data augmentation strategy to enhance feature extraction by leveraging various characteristics of time series data.
  arXiv  Detail & Related papers  (2025-03-28T07:59:23Z)
• Forward-Backward Knowledge Distillation for Continual Clustering [14.234785944941672]
  Unsupervised Continual Learning (UCL) is a burgeoning field in machine learning, focusing on enabling neural networks to sequentially learn tasks without explicit label information. Catastrophic Forgetting (CF) poses a significant challenge in continual learning, especially in UCL, where labeled information of data is not accessible. We introduce the concept of Unsupervised Continual Clustering (UCC), demonstrating enhanced performance and memory efficiency in clustering across various tasks.
  arXiv  Detail & Related papers  (2024-05-29T16:13:54Z)
• Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion [50.03041373044267]
  We propose a Supervised Contrastive learning framework with adaptive classification criterion for Continual Learning. Experiments show that CFL achieves state-of-the-art performance and has a stronger ability to overcome compared with the classification baselines.
  arXiv  Detail & Related papers  (2023-05-20T19:22:40Z)
• Beyond Supervised Continual Learning: a Review [69.9674326582747]
  Continual Learning (CL) is a flavor of machine learning where the usual assumption of stationary data distribution is relaxed or omitted. Changes in the data distribution can cause the so-called catastrophic forgetting (CF) effect: an abrupt loss of previous knowledge. This article reviews literature that study CL in other settings, such as learning with reduced supervision, fully unsupervised learning, and reinforcement learning.
  arXiv  Detail & Related papers  (2022-08-30T14:44:41Z)
• Online Continual Learning with Contrastive Vision Transformer [67.72251876181497]
  This paper proposes a framework Contrastive Vision Transformer (CVT) to achieve a better stability-plasticity trade-off for online CL. Specifically, we design a new external attention mechanism for online CL that implicitly captures previous tasks' information. Based on the learnable focuses, we design a focal contrastive loss to rebalance contrastive learning between new and past classes and consolidate previously learned representations.
  arXiv  Detail & Related papers  (2022-07-24T08:51:02Z)
• You Never Cluster Alone [150.94921340034688]
  We extend the mainstream contrastive learning paradigm to a cluster-level scheme, where all the data subjected to the same cluster contribute to a unified representation. We define a set of categorical variables as clustering assignment confidence, which links the instance-level learning track with the cluster-level one. By reparametrizing the assignment variables, TCC is trained end-to-end, requiring no alternating steps.
  arXiv  Detail & Related papers  (2021-06-03T14:59:59Z)
• Continual Learning From Unlabeled Data Via Deep Clustering [7.704949298975352]
  Continual learning aims to learn new tasks incrementally using less computation and memory resources instead of retraining the model from scratch whenever new task arrives. We introduce a new framework to make continual learning feasible in unsupervised mode by using pseudo label obtained from cluster assignments to update model.
  arXiv  Detail & Related papers  (2021-04-14T23:46:17Z)
• ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning [52.831894583501395]
  Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications. We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN) We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
  arXiv  Detail & Related papers  (2021-01-02T09:04:14Z)
• Online Deep Clustering for Unsupervised Representation Learning [108.33534231219464]
  Online Deep Clustering (ODC) performs clustering and network update simultaneously rather than alternatingly. We design and maintain two dynamic memory modules, i.e., samples memory to store samples labels and features, and centroids memory for centroids evolution. In this way, labels and the network evolve shoulder-to-shoulder rather than alternatingly.
  arXiv  Detail & Related papers  (2020-06-18T16:15:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.