Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective

• URL: http://arxiv.org/abs/2412.08285v5
• Date: Sat, 18 Jan 2025 08:33:38 GMT
• Title: Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective
• Authors: Minh Le, Tien Ngoc Luu, An Nguyen The, Thanh-Thien Le, Trang Nguyen, Tung Thanh Nguyen, Linh Ngo Van, Thien Huu Nguyen,
• Abstract summary: We propose a novel approach to address catastrophic forgetting in Continual Relation Extraction.<n>Our approach employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances.
• Score: 23.79259400522239
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To address catastrophic forgetting in Continual Relation Extraction (CRE), many current approaches rely on memory buffers to rehearse previously learned knowledge while acquiring new tasks. Recently, prompt-based methods have emerged as potent alternatives to rehearsal-based strategies, demonstrating strong empirical performance. However, upon analyzing existing prompt-based approaches for CRE, we identified several critical limitations, such as inaccurate prompt selection, inadequate mechanisms for mitigating forgetting in shared parameters, and suboptimal handling of cross-task and within-task variances. To overcome these challenges, we draw inspiration from the relationship between prefix-tuning and mixture of experts, proposing a novel approach that employs a prompt pool for each task, capturing variations within each task while enhancing cross-task variances. Furthermore, we incorporate a generative model to consolidate prior knowledge within shared parameters, eliminating the need for explicit data storage. Extensive experiments validate the efficacy of our approach, demonstrating superior performance over state-of-the-art prompt-based and rehearsal-free methods in continual relation extraction.

Related papers

• Is Parameter Isolation Better for Prompt-Based Continual Learning? [46.254917907419895]
  Most existing methods assign a fixed set of prompts to each task, isolating knowledge across tasks and resulting in suboptimal parameter utilization.<n>This framework constructs a global prompt pool and introduces a task-aware gated routing mechanism that sparsely activates a subset of prompts.<n>We also introduce a history-aware modulator that leverages cumulative prompt activation statistics to protect frequently used prompts from excessive updates.
  arXiv  Detail & Related papers  (2026-01-28T08:17:11Z)
• Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype [28.631643441543574]
  Continual learning aims to enable models to acquire new knowledge while retaining previously learned information.<n>We propose a novel approach employing task-specific Prompt-Prototype (ProP)<n>In our method, task-specific prompts facilitate more effective feature learning for the current task, while corresponding prototypes capture the representative features of the input.
  arXiv  Detail & Related papers  (2026-01-08T11:59:35Z)
• Forget Less, Retain More: A Lightweight Regularizer for Rehearsal-Based Continual Learning [51.07663354001582]
  Deep neural networks suffer from catastrophic forgetting, where performance on previous tasks degrades after training on a new task.<n>We present a novel approach to address this challenge, focusing on the intersection of memory-based methods and regularization approaches.<n>We formulate a regularization strategy, termed Information Maximization (IM) regularizer, for memory-based continual learning methods.
  arXiv  Detail & Related papers  (2025-12-01T15:56:00Z)
• One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning [52.966712416640085]
  We propose SMoPE, a novel framework that integrates the benefits of both task-specific and shared prompt strategies.<n>SMoPE consistently outperforms task-specific prompt methods and achieves performance competitive with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-09-29T08:54:58Z)
• Towards Rehearsal-Free Continual Relation Extraction: Capturing Within-Task Variance with Adaptive Prompting [2.818102173042532]
  WAVE++ is a novel approach inspired by the connection between prefix-tuning and mixture of experts.<n>We introduce task-specific prompt pools that enhance flexibility and adaptability across diverse tasks.<n>We incorporate label descriptions that provide richer, more global context, enabling the model to better distinguish among different relations.
  arXiv  Detail & Related papers  (2025-05-20T05:22:17Z)
• Few-Shot, No Problem: Descriptive Continual Relation Extraction [27.296604792388646]
  Few-shot Continual Relation Extraction is a crucial challenge for enabling AI systems to identify and adapt to evolving relationships in real-world domains. Traditional memory-based approaches often overfit to limited samples, failing to reinforce old knowledge. We propose a novel retrieval-based solution, starting with a large language model to generate descriptions for each relation.
  arXiv  Detail & Related papers  (2025-02-27T23:44:30Z)
• Diversified Sampling Improves Scaling LLM inference [31.18762591875725]
  DivSampling is a novel and versatile sampling technique designed to enhance the diversity of candidate solutions. Our theoretical analysis demonstrates that, under mild assumptions, the error rates of responses generated from diverse prompts are significantly lower compared to those produced by stationary prompts.
  arXiv  Detail & Related papers  (2025-02-16T07:37:58Z)
• TemPrompt: Multi-Task Prompt Learning for Temporal Relation Extraction in RAG-based Crowdsourcing Systems [21.312052922118585]
  Temporal relation extraction (TRE) aims to grasp the evolution of events or actions, and thus shape the workflow of associated tasks. We propose a multi-task prompt learning framework for TRE (TemPrompt), incorporating prompt tuning and contrastive learning to tackle these issues.
  arXiv  Detail & Related papers  (2024-06-21T01:52:37Z)
• Semantic Prompting with Image-Token for Continual Learning [7.5140668729696145]
  I-Prompt is a task-agnostic approach to eliminate task prediction. Our method achieves competitive performance on four benchmarks. We demonstrate the superiority of our method across various scenarios through extensive experiments.
  arXiv  Detail & Related papers  (2024-03-18T07:43:14Z)
• Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model [86.9619638550683]
  Vision-language foundation models have exhibited remarkable success across a multitude of downstream tasks due to their scalability on extensive image-text paired data. However, these models display significant limitations when applied to downstream tasks, such as fine-grained image classification, as a result of decision shortcuts''
  arXiv  Detail & Related papers  (2024-03-01T09:01:53Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• Similarity-based Memory Enhanced Joint Entity and Relation Extraction [3.9659135716762894]
  Document-level joint entity and relation extraction is a challenging information extraction problem. We present a multi-task learning framework with bidirectional memory-like dependency between tasks. Our empirical studies show that the proposed approach outperforms the existing methods.
  arXiv  Detail & Related papers  (2023-07-14T12:26:56Z)
• FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment [93.09267863425492]
  We argue that understanding both high-level semantics and internal temporal structures of actions in competitive sports videos is the key to making predictions accurate and interpretable. We construct a new fine-grained dataset, called FineDiving, developed on diverse diving events with detailed annotations on action procedures.
  arXiv  Detail & Related papers  (2022-04-07T17:59:32Z)
• Variational Multi-Task Learning with Gumbel-Softmax Priors [105.22406384964144]
  Multi-task learning aims to explore task relatedness to improve individual tasks. We propose variational multi-task learning (VMTL), a general probabilistic inference framework for learning multiple related tasks.
  arXiv  Detail & Related papers  (2021-11-09T18:49:45Z)
• An Investigation of Replay-based Approaches for Continual Learning [79.0660895390689]
  Continual learning (CL) is a major challenge of machine learning (ML) and describes the ability to learn several tasks sequentially without catastrophic forgetting (CF) Several solution classes have been proposed, of which so-called replay-based approaches seem very promising due to their simplicity and robustness. We empirically investigate replay-based approaches of continual learning and assess their potential for applications.
  arXiv  Detail & Related papers  (2021-08-15T15:05:02Z)
• Learning Invariant Representation for Continual Learning [5.979373021392084]
  A key challenge in Continual learning is catastrophically forgetting previously learned tasks when the agent faces a new one. We propose a new pseudo-rehearsal-based method, named learning Invariant Representation for Continual Learning (IRCL) Disentangling the shared invariant representation helps to learn continually a sequence of tasks, while being more robust to forgetting and having better knowledge transfer.
  arXiv  Detail & Related papers  (2021-01-15T15:12:51Z)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction [166.87111665908333]
  We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL) Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
  arXiv  Detail & Related papers  (2020-04-29T02:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.