Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning

• URL: http://arxiv.org/abs/2109.06349v1
• Date: Mon, 13 Sep 2021 22:28:58 GMT
• Title: Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning
• Authors: Jianguo Zhang, Trung Bui, Seunghyun Yoon, Xiang Chen, Zhiwei Liu, Congying Xia, Quan Hung Tran, Walter Chang, Philip Yu
• Abstract summary: We present a simple yet effective few-shot intent detection schema via contrastive pre-training and fine-tuning. We first conduct self-supervised contrastive pre-training on collected intent datasets, which implicitly learns to discriminate semantically similar utterances. We then perform few-shot intent detection together with supervised contrastive learning, which explicitly pulls utterances from the same intent closer.
• Score: 27.154414939086426
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we focus on a more challenging few-shot intent detection scenario where many intents are fine-grained and semantically similar. We present a simple yet effective few-shot intent detection schema via contrastive pre-training and fine-tuning. Specifically, we first conduct self-supervised contrastive pre-training on collected intent datasets, which implicitly learns to discriminate semantically similar utterances without using any labels. We then perform few-shot intent detection together with supervised contrastive learning, which explicitly pulls utterances from the same intent closer and pushes utterances across different intents farther. Experimental results show that our proposed method achieves state-of-the-art performance on three challenging intent detection datasets under 5-shot and 10-shot settings.

Related papers

• All Labels Together: Low-shot Intent Detection with an Efficient Label Semantic Encoding Paradigm [48.02790193676742]
  In intent detection tasks, leveraging meaningful semantic information from intent labels can be particularly beneficial for few-shot scenarios. We present an end-to-end One-to-All system that enables the comparison of an input utterance with all label candidates. Experiments on three few-shot intent detection tasks demonstrate that One-to-All is especially effective when the training resource is extremely scarce.
  arXiv  Detail & Related papers  (2023-09-07T08:50:45Z)
• QAID: Question Answering Inspired Few-shot Intent Detection [5.516275800944541]
  We reformulate intent detection as a question-answering retrieval task by treating utterances and intent names as questions and answers. Our results on three few-shot intent detection benchmarks achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-03-02T21:35:15Z)
• Learning Discriminative Representations and Decision Boundaries for Open Intent Detection [16.10123071366136]
  Open intent detection is a significant problem in natural language understanding. We propose DA-ADB, which learns distance-aware intent representations and adaptive decision boundaries for open intent detection. Our framework achieves substantial improvements on three benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-11T10:02:09Z)
• Exploring the Limits of Natural Language Inference Based Setup for Few-Shot Intent Detection [13.971616443394474]
  Generalized Few-shot intent detection is more realistic but challenging setup. We employ a simple and effective method based on Natural Language Inference. Our method achieves state-of-the-art results on 1-shot and 5-shot intent detection task.
  arXiv  Detail & Related papers  (2021-12-14T14:47:23Z)
• Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learning [51.03781020616402]
  Fine-grained action recognition is attracting increasing attention due to the emerging demand of specific action understanding in real-world applications. We propose a few-shot fine-grained action recognition problem, aiming to recognize novel fine-grained actions with only few samples given for each class. Although progress has been made in coarse-grained actions, existing few-shot recognition methods encounter two issues handling fine-grained actions.
  arXiv  Detail & Related papers  (2021-08-15T02:21:01Z)
• Incremental False Negative Detection for Contrastive Learning [95.68120675114878]
  We introduce a novel incremental false negative detection for self-supervised contrastive learning. During contrastive learning, we discuss two strategies to explicitly remove the detected false negatives. Our proposed method outperforms other self-supervised contrastive learning frameworks on multiple benchmarks within a limited compute.
  arXiv  Detail & Related papers  (2021-06-07T15:29:14Z)
• Generalized Zero-shot Intent Detection via Commonsense Knowledge [5.398580049917152]
  We propose RIDE: an intent detection model that leverages commonsense knowledge in an unsupervised fashion to overcome the issue of training data scarcity. RIDE computes robust and generalizable relationship meta-features that capture deep semantic relationships between utterances and intent labels. Our extensive experimental analysis on three widely-used intent detection benchmarks shows that relationship meta-features significantly increase the accuracy of detecting both seen and unseen intents.
  arXiv  Detail & Related papers  (2021-02-04T23:36:41Z)
• Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference [150.07326223077405]
  Few-shot learning is attracting much attention to mitigate data scarcity. We present a discriminative nearest neighbor classification with deep self-attention. We propose to boost the discriminative ability by transferring a natural language inference (NLI) model.
  arXiv  Detail & Related papers  (2020-10-25T00:39:32Z)
• Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection [69.2370349274216]
  Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Semantic components are distilled from utterances via multi-head self-attention. Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
  arXiv  Detail & Related papers  (2020-10-06T05:16:38Z)
• Efficient Intent Detection with Dual Sentence Encoders [53.16532285820849]
  We introduce intent detection methods backed by pretrained dual sentence encoders such as USE and ConveRT. We demonstrate the usefulness and wide applicability of the proposed intent detectors, showing that they outperform intent detectors based on fine-tuning the full BERT-Large model. We release our code, as well as a new challenging single-domain intent detection dataset.
  arXiv  Detail & Related papers  (2020-03-10T15:33:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.