Prompt Learning With Knowledge Memorizing Prototypes For Generalized Few-Shot Intent Detection

• URL: http://arxiv.org/abs/2309.04971v1
• Date: Sun, 10 Sep 2023 09:16:38 GMT
• Title: Prompt Learning With Knowledge Memorizing Prototypes For Generalized Few-Shot Intent Detection
• Authors: Chaiyut Luoyiching, Yangning Li, Yinghui Li, Rongsheng Li, Hai-Tao Zheng, Nannan Zhou, Hanjing Su
• Abstract summary: Generalized Few-Shot Intent Detection (GFSID) is challenging and realistic because it needs to categorize both seen and novel intents simultaneously. Previous GFSID methods rely on the episodic learning paradigm. We propose to convert the GFSID task into the class incremental learning paradigm.
• Score: 22.653220906899612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generalized Few-Shot Intent Detection (GFSID) is challenging and realistic because it needs to categorize both seen and novel intents simultaneously. Previous GFSID methods rely on the episodic learning paradigm, which makes it hard to extend to a generalized setup as they do not explicitly learn the classification of seen categories and the knowledge of seen intents. To address the dilemma, we propose to convert the GFSID task into the class incremental learning paradigm. Specifically, we propose a two-stage learning framework, which sequentially learns the knowledge of different intents in various periods via prompt learning. And then we exploit prototypes for categorizing both seen and novel intents. Furthermore, to achieve the transfer knowledge of intents in different stages, for different scenarios we design two knowledge preservation methods which close to realistic applications. Extensive experiments and detailed analyses on two widely used datasets show that our framework based on the class incremental learning paradigm achieves promising performance.

Related papers

• Contextuality Helps Representation Learning for Generalized Category Discovery [5.885208652383516]
  This paper introduces a novel approach to Generalized Category Discovery (GCD) by leveraging the concept of contextuality. Our model integrates two levels of contextuality: instance-level, where nearest-neighbor contexts are utilized for contrastive learning, and cluster-level, employing contrastive learning. The integration of the contextual information effectively improves the feature learning and thereby the classification accuracy of all categories.
  arXiv  Detail & Related papers  (2024-07-29T07:30:41Z)
• Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning [50.1035273069458]
  Spoken language understanding (SLU) is a core task in task-oriented dialogue systems. We propose a multi-level MMCL framework to apply contrastive learning at three levels, including utterance level, slot level, and word level. Our framework achieves new state-of-the-art results on two public multi-intent SLU datasets.
  arXiv  Detail & Related papers  (2024-05-31T14:34:23Z)
• Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learning [20.34477942813382]
  Skeleton-based action representation learning aims to interpret and understand human behaviors by encoding the skeleton sequences. We introduce a novel skeleton-based training framework based on Cross-modal Contrastive learning. Our method outperforms the previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-05-31T03:40:15Z)
• Scalable Language Model with Generalized Continual Learning [58.700439919096155]
  The Joint Adaptive Re-ization (JARe) is integrated with Dynamic Task-related Knowledge Retrieval (DTKR) to enable adaptive adjustment of language models based on specific downstream tasks. Our method demonstrates state-of-the-art performance on diverse backbones and benchmarks, achieving effective continual learning in both full-set and few-shot scenarios with minimal forgetting.
  arXiv  Detail & Related papers  (2024-04-11T04:22:15Z)
• Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Models [64.24227572048075]
  We propose a Knowledge-Aware Prompt Tuning (KAPT) framework for vision-language models. Our approach takes inspiration from human intelligence in which external knowledge is usually incorporated into recognizing novel categories of objects.
  arXiv  Detail & Related papers  (2023-08-22T04:24:45Z)
• Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning [62.839109775887025]
  Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit unsupervised semantic knowledge extracted from PLM. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:08:57Z)
• Novel Class Discovery without Forgetting [72.52222295216062]
  We identify and formulate a new, pragmatic problem setting of NCDwF: Novel Class Discovery without Forgetting. We propose a machine learning model to incrementally discover novel categories of instances from unlabeled data. We introduce experimental protocols based on CIFAR-10, CIFAR-100 and ImageNet-1000 to measure the trade-off between knowledge retention and novel class discovery.
  arXiv  Detail & Related papers  (2022-07-21T17:54:36Z)
• Learning to Prompt for Continual Learning [34.609384246149325]
  This work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity.
  arXiv  Detail & Related papers  (2021-12-16T06:17:07Z)
• Learning Disentangled Intent Representations for Zero-shot Intent Detection [13.19024497857648]
  We propose a class-transductive framework that utilizes unseen class labels to learn Disentangled Representations (DIR) Under this framework, we introduce a multi-task learning objective, which encourages the model to learn the distinctions among intents. Experiments on two real-world datasets show that the proposed framework brings consistent improvement to the baseline systems.
  arXiv  Detail & Related papers  (2020-12-03T06:41:09Z)
• Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification [106.08067870620218]
  We propose a self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME) We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model. We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-01-06T12:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.