Prompt Learning With Knowledge Memorizing Prototypes For Generalized Few-Shot Intent Detection

• URL: http://arxiv.org/abs/2309.04971v1
• Date: Sun, 10 Sep 2023 09:16:38 GMT
• Title: Prompt Learning With Knowledge Memorizing Prototypes For Generalized Few-Shot Intent Detection
• Authors: Chaiyut Luoyiching, Yangning Li, Yinghui Li, Rongsheng Li, Hai-Tao Zheng, Nannan Zhou, Hanjing Su
• Abstract summary: Generalized Few-Shot Intent Detection (GFSID) is challenging and realistic because it needs to categorize both seen and novel intents simultaneously. Previous GFSID methods rely on the episodic learning paradigm. We propose to convert the GFSID task into the class incremental learning paradigm.
• Score: 22.653220906899612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalized Few-Shot Intent Detection (GFSID) is challenging and realistic because it needs to categorize both seen and novel intents simultaneously. Previous GFSID methods rely on the episodic learning paradigm, which makes it hard to extend to a generalized setup as they do not explicitly learn the classification of seen categories and the knowledge of seen intents. To address the dilemma, we propose to convert the GFSID task into the class incremental learning paradigm. Specifically, we propose a two-stage learning framework, which sequentially learns the knowledge of different intents in various periods via prompt learning. And then we exploit prototypes for categorizing both seen and novel intents. Furthermore, to achieve the transfer knowledge of intents in different stages, for different scenarios we design two knowledge preservation methods which close to realistic applications. Extensive experiments and detailed analyses on two widely used datasets show that our framework based on the class incremental learning paradigm achieves promising performance.

Related papers

• CSTA: Spatial-Temporal Causal Adaptive Learning for Exemplar-Free Video Class-Incremental Learning [62.69917996026769]
  A class-incremental learning task requires learning and preserving both spatial appearance and temporal action involvement. We propose a framework that equips separate adapters to learn new class patterns, accommodating the incremental information requirements unique to each class. A causal compensation mechanism is proposed to reduce the conflicts during increment and memorization for between different types of information.
  arXiv  Detail & Related papers  (2025-01-13T11:34:55Z)
• SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization [2.1682783789464968]
  Fine-grained Action Recognition (FAR) focuses on detailed semantic labels within shorter temporal duration. Given the high costs of annotating labels and the substantial data needed for fine-tuning LLMs, we propose to adopt semi-supervised learning (SSL) Our framework, SeFAR, incorporates several innovative designs to tackle these challenges.
  arXiv  Detail & Related papers  (2025-01-02T13:12:12Z)
• Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning [50.1035273069458]
  Spoken language understanding (SLU) is a core task in task-oriented dialogue systems. We propose a multi-level MMCL framework to apply contrastive learning at three levels, including utterance level, slot level, and word level. Our framework achieves new state-of-the-art results on two public multi-intent SLU datasets.
  arXiv  Detail & Related papers  (2024-05-31T14:34:23Z)
• Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learning [20.34477942813382]
  Skeleton-based action representation learning aims to interpret and understand human behaviors by encoding the skeleton sequences. We introduce a novel skeleton-based training framework based on Cross-modal Contrastive learning. Our method outperforms the previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-05-31T03:40:15Z)
• Scalable Language Model with Generalized Continual Learning [58.700439919096155]
  The Joint Adaptive Re-ization (JARe) is integrated with Dynamic Task-related Knowledge Retrieval (DTKR) to enable adaptive adjustment of language models based on specific downstream tasks. Our method demonstrates state-of-the-art performance on diverse backbones and benchmarks, achieving effective continual learning in both full-set and few-shot scenarios with minimal forgetting.
  arXiv  Detail & Related papers  (2024-04-11T04:22:15Z)
• Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Models [64.24227572048075]
  We propose a Knowledge-Aware Prompt Tuning (KAPT) framework for vision-language models. Our approach takes inspiration from human intelligence in which external knowledge is usually incorporated into recognizing novel categories of objects.
  arXiv  Detail & Related papers  (2023-08-22T04:24:45Z)
• Novel Class Discovery without Forgetting [72.52222295216062]
  We identify and formulate a new, pragmatic problem setting of NCDwF: Novel Class Discovery without Forgetting. We propose a machine learning model to incrementally discover novel categories of instances from unlabeled data. We introduce experimental protocols based on CIFAR-10, CIFAR-100 and ImageNet-1000 to measure the trade-off between knowledge retention and novel class discovery.
  arXiv  Detail & Related papers  (2022-07-21T17:54:36Z)
• Learning to Prompt for Continual Learning [34.609384246149325]
  This work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity.
  arXiv  Detail & Related papers  (2021-12-16T06:17:07Z)
• Learning Disentangled Intent Representations for Zero-shot Intent Detection [13.19024497857648]
  We propose a class-transductive framework that utilizes unseen class labels to learn Disentangled Representations (DIR) Under this framework, we introduce a multi-task learning objective, which encourages the model to learn the distinctions among intents. Experiments on two real-world datasets show that the proposed framework brings consistent improvement to the baseline systems.
  arXiv  Detail & Related papers  (2020-12-03T06:41:09Z)
• Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification [106.08067870620218]
  We propose a self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME) We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model. We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-01-06T12:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.