Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype

• URL: http://arxiv.org/abs/2601.04864v1
• Date: Thu, 08 Jan 2026 11:59:35 GMT
• Title: Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype
• Authors: Haihua Luo, Xuming Ran, Zhengji Li, Huiyan Xue, Tingting Jiang, Jiangrong Shen, Tommi Kärkkäinen, Qi Xu, Fengyu Cong,
• Abstract summary: Continual learning aims to enable models to acquire new knowledge while retaining previously learned information.<n>We propose a novel approach employing task-specific Prompt-Prototype (ProP)<n>In our method, task-specific prompts facilitate more effective feature learning for the current task, while corresponding prototypes capture the representative features of the input.
• Score: 28.631643441543574
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Continual learning aims to enable models to acquire new knowledge while retaining previously learned information. Prompt-based methods have shown remarkable performance in this domain; however, they typically rely on key-value pairing, which can introduce inter-task interference and hinder scalability. To overcome these limitations, we propose a novel approach employing task-specific Prompt-Prototype (ProP), thereby eliminating the need for key-value pairs. In our method, task-specific prompts facilitate more effective feature learning for the current task, while corresponding prototypes capture the representative features of the input. During inference, predictions are generated by binding each task-specific prompt with its associated prototype. Additionally, we introduce regularization constraints during prompt initialization to penalize excessively large values, thereby enhancing stability. Experiments on several widely used datasets demonstrate the effectiveness of the proposed method. In contrast to mainstream prompt-based approaches, our framework removes the dependency on key-value pairs, offering a fresh perspective for future continual learning research.

Related papers

• Is Parameter Isolation Better for Prompt-Based Continual Learning? [46.254917907419895]
  Most existing methods assign a fixed set of prompts to each task, isolating knowledge across tasks and resulting in suboptimal parameter utilization.<n>This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts.<n>We also introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates.
  arXiv  Detail & Related papers  (2026-01-28T08:17:11Z)
• All You Need is One: Capsule Prompt Tuning with a Single Vector [86.68105855537762]
  Current prompt-based learning methods rely on laborious grid searching for optimal prompt length and typically require considerable number of prompts.<n>We introduce Capsule Prompt-Tuning (CaPT), an efficient and effective solution that leverages off-the-shelf, informative instance semantics into prompt-based learning.<n>Our approach innovatively integrates both instance-aware and task-aware information in a nearly parameter-free manner.
  arXiv  Detail & Related papers  (2025-10-19T00:02:59Z)
• One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning [52.966712416640085]
  We propose SMoPE, a novel framework that integrates the benefits of both task-specific and shared prompt strategies.<n>SMoPE consistently outperforms task-specific prompt methods and achieves performance competitive with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-09-29T08:54:58Z)
• Towards Rehearsal-Free Continual Relation Extraction: Capturing Within-Task Variance with Adaptive Prompting [2.818102173042532]
  WAVE++ is a novel approach inspired by the connection between prefix-tuning and mixture of experts.<n>We introduce task-specific prompt pools that enhance flexibility and adaptability across diverse tasks.<n>We incorporate label descriptions that provide richer, more global context, enabling the model to better distinguish among different relations.
  arXiv  Detail & Related papers  (2025-05-20T05:22:17Z)
• Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective [23.79259400522239]
  We propose a novel approach to address catastrophic forgetting in Continual Relation Extraction.<n>Our approach employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances.
  arXiv  Detail & Related papers  (2024-12-11T11:00:33Z)
• Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model [86.9619638550683]
  Vision-language foundation models have exhibited remarkable success across a multitude of downstream tasks due to their scalability on extensive image-text paired data.<n>However, these models display significant limitations when applied to downstream tasks, such as fine-grained image classification, as a result of decision shortcuts''
  arXiv  Detail & Related papers  (2024-03-01T09:01:53Z)
• KOPPA: Improving Prompt-based Continual Learning with Key-Query Orthogonal Projection and Prototype-based One-Versus-All [24.50129285997307]
  We introduce a novel key-query learning strategy to enhance prompt matching efficiency and address the challenge of shifting features. Our method empowers the model to achieve results surpassing those of current state-of-the-art approaches by a large margin of up to 20%.
  arXiv  Detail & Related papers  (2023-11-26T20:35:19Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Instance-wise Prompt Tuning for Pretrained Language Models [72.74916121511662]
  Instance-wise Prompt Tuning (IPT) is the first prompt learning paradigm that injects knowledge from the input data instances to the prompts. IPT significantly outperforms task-based prompt learning methods, and achieves comparable performance to conventional finetuning with only 0.5% - 1.5% of tuned parameters.
  arXiv  Detail & Related papers  (2022-06-04T10:08:50Z)
• Continual Prompt Tuning for Dialog State Tracking [58.66412648276873]
  A desirable dialog system should be able to continually learn new skills without forgetting old ones. We present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks.
  arXiv  Detail & Related papers  (2022-03-13T13:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.