KAC: Kolmogorov-Arnold Classifier for Continual Learning

• URL: http://arxiv.org/abs/2503.21076v1
• Date: Thu, 27 Mar 2025 01:27:14 GMT
• Title: KAC: Kolmogorov-Arnold Classifier for Continual Learning
• Authors: Yusong Hu, Zichen Liang, Fei Yang, Qibin Hou, Xialei Liu, Ming-Ming Cheng,
• Abstract summary: Continual learning requires models to train continuously across consecutive tasks without forgetting.<n>Most existing methods utilize linear classifiers, which struggle to maintain a stable classification space while learning new tasks.<n>Inspired by the success of Kolmogorov-Arnold Networks (KAN) in preserving learning during simple continual regression tasks, we set out to explore their potential in more complex continual learning scenarios.
• Score: 70.29494592027852
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning requires models to train continuously across consecutive tasks without forgetting. Most existing methods utilize linear classifiers, which struggle to maintain a stable classification space while learning new tasks. Inspired by the success of Kolmogorov-Arnold Networks (KAN) in preserving learning stability during simple continual regression tasks, we set out to explore their potential in more complex continual learning scenarios. In this paper, we introduce the Kolmogorov-Arnold Classifier (KAC), a novel classifier developed for continual learning based on the KAN structure. We delve into the impact of KAN's spline functions and introduce Radial Basis Functions (RBF) for improved compatibility with continual learning. We replace linear classifiers with KAC in several recent approaches and conduct experiments across various continual learning benchmarks, all of which demonstrate performance improvements, highlighting the effectiveness and robustness of KAC in continual learning. The code is available at https://github.com/Ethanhuhuhu/KAC.

Related papers

• STAR: Stability-Inducing Weight Perturbation for Continual Learning [4.623295991242981]
  A key challenge in continual learning is that as the model is updated to learn new tasks, it becomes susceptible to catastrophic forgetting.<n>A popular approach to mitigate forgetting during continual learning is to maintain a small buffer of previously-seen samples and to replay them during training.<n>We propose a novel loss function, STAR, that exploits the worst-case parameter perturbation that reduces the KL-divergence of model predictions.
  arXiv  Detail & Related papers  (2025-03-03T14:32:03Z)
• Continual Task Learning through Adaptive Policy Self-Composition [54.95680427960524]
  CompoFormer is a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Our experiments reveal that CompoFormer outperforms conventional continual learning (CL) methods, particularly in longer task sequences.
  arXiv  Detail & Related papers  (2024-11-18T08:20:21Z)
• Temporal-Difference Variational Continual Learning [89.32940051152782]
  A crucial capability of Machine Learning models in real-world applications is the ability to continuously learn new tasks. In Continual Learning settings, models often struggle to balance learning new tasks with retaining previous knowledge. We propose new learning objectives that integrate the regularization effects of multiple previous posterior estimations.
  arXiv  Detail & Related papers  (2024-10-10T10:58:41Z)
• Densely Distilling Cumulative Knowledge for Continual Learning [14.343655566551213]
  Continual learning, involving sequential training on diverse tasks, often faces catastrophic forgetting. We propose Dense Knowledge Distillation (DKD) to distill the cumulative knowledge of all the previous tasks. Our DKD outperforms recent state-of-the-art baselines across diverse benchmarks and scenarios.
  arXiv  Detail & Related papers  (2024-05-16T05:37:06Z)
• Auxiliary Classifiers Improve Stability and Efficiency in Continual Learning [13.309853617922824]
  We investigate the stability of intermediate neural network layers during continual learning. We show auxiliary classifiers (ACs) can leverage this stability to improve performance. Our findings suggest that ACs offer a promising avenue for enhancing continual learning models.
  arXiv  Detail & Related papers  (2024-03-12T08:33:26Z)
• COOLer: Class-Incremental Learning for Appearance-Based Multiple Object Tracking [32.47215340215641]
  This paper extends the scope of continual learning research to class-incremental learning for multiple object tracking (MOT) Previous solutions for continual learning of object detectors do not address the data association stage of appearance-based trackers. We introduce COOLer, a COntrastive- and cOntinual-Learning-based tracker, which incrementally learns to track new categories while preserving past knowledge.
  arXiv  Detail & Related papers  (2023-10-04T17:49:48Z)
• Mitigating Catastrophic Forgetting in Task-Incremental Continual Learning with Adaptive Classification Criterion [50.03041373044267]
  We propose a Supervised Contrastive learning framework with adaptive classification criterion for Continual Learning. Experiments show that CFL achieves state-of-the-art performance and has a stronger ability to overcome compared with the classification baselines.
  arXiv  Detail & Related papers  (2023-05-20T19:22:40Z)
• Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning [54.7584721943286]
  Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal. We propose to only activate and select sparse neurons for learning current and past tasks at any stage.
  arXiv  Detail & Related papers  (2022-02-21T13:25:03Z)
• Task-agnostic Continual Learning with Hybrid Probabilistic Models [75.01205414507243]
  We propose HCL, a Hybrid generative-discriminative approach to Continual Learning for classification. The flow is used to learn the data distribution, perform classification, identify task changes, and avoid forgetting. We demonstrate the strong performance of HCL on a range of continual learning benchmarks such as split-MNIST, split-CIFAR, and SVHN-MNIST.
  arXiv  Detail & Related papers  (2021-06-24T05:19:26Z)
• Continual Learning From Unlabeled Data Via Deep Clustering [7.704949298975352]
  Continual learning aims to learn new tasks incrementally using less computation and memory resources instead of retraining the model from scratch whenever new task arrives. We introduce a new framework to make continual learning feasible in unsupervised mode by using pseudo label obtained from cluster assignments to update model.
  arXiv  Detail & Related papers  (2021-04-14T23:46:17Z)
• Continual Learning with Node-Importance based Adaptive Group Sparse Regularization [30.23319528662881]
  We propose a novel regularization-based continual learning method, dubbed as Adaptive Group Sparsity based Continual Learning (AGS-CL) Our method selectively employs the two penalties when learning each node based its the importance, which is adaptively updated after learning each new task.
  arXiv  Detail & Related papers  (2020-03-30T18:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.