Advancing Prompt-Based Methods for Replay-Independent General Continual Learning

• URL: http://arxiv.org/abs/2503.00677v1
• Date: Sun, 02 Mar 2025 00:58:18 GMT
• Title: Advancing Prompt-Based Methods for Replay-Independent General Continual Learning
• Authors: Zhiqi Kang, Liyuan Wang, Xingxing Zhang, Karteek Alahari,
• Abstract summary: General continual learning (GCL) is a broad concept to describe real-world continual learning (CL) problems.<n>Such requirements result in poor initial performance, limited generalizability, and severe catastrophic forgetting.<n>We propose an innovative approach named MISA (Mask and Initial Session Adaption) to advance prompt-based methods in GCL.
• Score: 44.94466949172424
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: General continual learning (GCL) is a broad concept to describe real-world continual learning (CL) problems, which are often characterized by online data streams without distinct transitions between tasks, i.e., blurry task boundaries. Such requirements result in poor initial performance, limited generalizability, and severe catastrophic forgetting, heavily impacting the effectiveness of mainstream GCL models trained from scratch. While the use of a frozen pretrained backbone with appropriate prompt tuning can partially address these challenges, such prompt-based methods remain suboptimal for CL of remaining tunable parameters on the fly. In this regard, we propose an innovative approach named MISA (Mask and Initial Session Adaption) to advance prompt-based methods in GCL. It includes a forgetting-aware initial session adaption that employs pretraining data to initialize prompt parameters and improve generalizability, as well as a non-parametric logit mask of the output layers to mitigate catastrophic forgetting. Empirical results demonstrate substantial performance gains of our approach compared to recent competitors, especially without a replay buffer (e.g., up to 18.39%, 22.06%, and 11.96% performance lead on CIFAR-100, Tiny-ImageNet, and ImageNet-R, respectively). Moreover, our approach features the plug-in nature for prompt-based methods, independence of replay, ease of implementation, and avoidance of CL-relevant hyperparameters, serving as a strong baseline for GCL research. Our source code is publicly available at https://github.com/kangzhiq/MISA

Related papers

• Fishing For Cheap And Efficient Pruners At Initialization [4.433137726540548]
  Pruning offers a promising solution to mitigate the associated costs and environmental impact of deploying large deep neural networks (DNNs)<n>We introduce Fisher-Taylor Sensitivity (FTS), a computationally cheap and efficient pruning criterion based on the empirical Fisher Information Matrix (FIM) diagonal.<n>Our method achieves competitive performance against state-of-the-art techniques for one-shot PBT, even under extreme sparsity conditions.
  arXiv  Detail & Related papers  (2025-02-17T05:22:23Z)
• REINFORCE++: An Efficient RLHF Algorithm with Robustness to Both Prompt and Reward Models [8.587685197004097]
  REINFORCE++ is a novel approach that removes the critic model while using the normalized reward of a batch as the baseline. It exhibits robust performance across various reward models without requiring prompt set truncation. It achieves superior generalization in both RLHF and long chain-of-thought settings compared to existing REINFORCE-based methods.
  arXiv  Detail & Related papers  (2025-01-04T02:08:06Z)
• TS-ACL: A Time Series Analytic Continual Learning Framework for Privacy-Preserving and Class-Incremental Pattern Recognition [14.108911377558242]
  TS-ACL is a novel framework for privacy-preserving and class-incremental pattern recognition. It transforms each update of the model into a gradient-free analytical learning process with a closed-form solution. It simultaneously achieves non-forgetting, privacy preservation, and lightweight consumption, making it widely suitable for various applications.
  arXiv  Detail & Related papers  (2024-10-21T12:34:02Z)
• SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning [63.93193829913252]
  We propose an innovative METL strategy called SHERL for resource-limited scenarios. In the early route, intermediate outputs are consolidated via an anti-redundancy operation. In the late route, utilizing minimal late pre-trained layers could alleviate the peak demand on memory overhead.
  arXiv  Detail & Related papers  (2024-07-10T10:22:35Z)
• Hyperparameters in Continual Learning: A Reality Check [53.30082523545212]
  Continual learning (CL) aims to train a model on a sequence of tasks while balancing the trade-off between plasticity (learning new tasks) and stability (retaining prior knowledge)
  arXiv  Detail & Related papers  (2024-03-14T03:13:01Z)
• Continual Learning with Dynamic Sparse Training: Exploring Algorithms for Effective Model Updates [13.983410740333788]
  Continual learning (CL) refers to the ability of an intelligent system to sequentially acquire and retain knowledge from a stream of data with as little computational overhead as possible. Dynamic Sparse Training (DST) is a prominent way to find these sparse networks and isolate them for each task. This paper is the first empirical study investigating the effect of different DST components under the CL paradigm.
  arXiv  Detail & Related papers  (2023-08-28T18:31:09Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)
• Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selection [59.77647907277523]
  Adversarial contrast learning (ACL) does not require expensive data annotations but outputs a robust representation that withstands adversarial attacks. ACL needs tremendous running time to generate the adversarial variants of all training data. This paper proposes a robustness-aware coreset selection (RCS) method to speed up ACL.
  arXiv  Detail & Related papers  (2023-02-08T03:20:14Z)
• GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning [1.911678487931003]
  We show significant gains (2%-4% absolute) over the state-of-the-art in the well-studied offline continual learning setting. Our findings also effectively transfer to online / streaming CL settings, showing upto 5% gains over existing approaches.
  arXiv  Detail & Related papers  (2021-11-18T18:01:57Z)
• Continual Learning in Recurrent Neural Networks [67.05499844830231]
  We evaluate the effectiveness of continual learning methods for processing sequential data with recurrent neural networks (RNNs) We shed light on the particularities that arise when applying weight-importance methods, such as elastic weight consolidation, to RNNs. We show that the performance of weight-importance methods is not directly affected by the length of the processed sequences, but rather by high working memory requirements.
  arXiv  Detail & Related papers  (2020-06-22T10:05:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.