IntentGPT: Few-shot Intent Discovery with Large Language Models

• URL: http://arxiv.org/abs/2411.10670v1
• Date: Sat, 16 Nov 2024 02:16:59 GMT
• Title: IntentGPT: Few-shot Intent Discovery with Large Language Models
• Authors: Juan A. Rodriguez, Nicholas Botzer, David Vazquez, Christopher Pal, Marco Pedersoli, Issam Laradji,
• Abstract summary: We develop a model capable of identifying new intents as they emerge. IntentGPT is a training-free method that effectively prompts Large Language Models (LLMs) to discover new intents with minimal labeled data. Our experiments show that IntentGPT outperforms previous methods that require extensive domain-specific data and fine-tuning.
• Score: 9.245106106117317
• License:
• Abstract: In today's digitally driven world, dialogue systems play a pivotal role in enhancing user interactions, from customer service to virtual assistants. In these dialogues, it is important to identify user's goals automatically to resolve their needs promptly. This has necessitated the integration of models that perform Intent Detection. However, users' intents are diverse and dynamic, making it challenging to maintain a fixed set of predefined intents. As a result, a more practical approach is to develop a model capable of identifying new intents as they emerge. We address the challenge of Intent Discovery, an area that has drawn significant attention in recent research efforts. Existing methods need to train on a substantial amount of data for correctly identifying new intents, demanding significant human effort. To overcome this, we introduce IntentGPT, a novel training-free method that effectively prompts Large Language Models (LLMs) such as GPT-4 to discover new intents with minimal labeled data. IntentGPT comprises an \textit{In-Context Prompt Generator}, which generates informative prompts for In-Context Learning, an \textit{Intent Predictor} for classifying and discovering user intents from utterances, and a \textit{Semantic Few-Shot Sampler} that selects relevant few-shot examples and a set of known intents to be injected into the prompt. Our experiments show that IntentGPT outperforms previous methods that require extensive domain-specific data and fine-tuning, in popular benchmarks, including CLINC and BANKING, among others.

Related papers

• Eliciting Human Preferences with Language Models [56.68637202313052]
  Language models (LMs) can be directed to perform target tasks by using labeled examples or natural language prompts. We propose to use *LMs themselves* to guide the task specification process. We study GATE in three domains: email validation, content recommendation, and moral reasoning.
  arXiv  Detail & Related papers  (2023-10-17T21:11:21Z)
• DialogVCS: Robust Natural Language Understanding in Dialogue System Upgrade [36.433020605744986]
  In constant updates of product dialogue systems, new data from the real users would be merged into the existent data. New intents would emerge and might have semantic entanglement with the existing intents. We setup a new benchmark consisting of 4 Dialogue Version Control dataSets (VCS) We formulate the intent detection with imperfect data in the system update as a multi-label classification task with positive but unlabeled intents.
  arXiv  Detail & Related papers  (2023-05-24T05:53:38Z)
• Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models [116.25562358482962]
  State-of-the-art neural language models can be used to solve ad-hoc language tasks without the need for supervised training. PromptIDE allows users to experiment with prompt variations, visualize prompt performance, and iteratively optimize prompts.
  arXiv  Detail & Related papers  (2022-08-16T17:17:53Z)
• New Intent Discovery with Pre-training and Contrastive Learning [21.25371293641141]
  New intent discovery aims to uncover novel intent categories from user utterances to expand the set of supported intent classes. Existing approaches typically rely on a large amount of labeled utterances. We propose a new contrastive loss to exploit self-supervisory signals in unlabeled data for clustering.
  arXiv  Detail & Related papers  (2022-05-25T17:07:25Z)
• A Framework to Generate High-Quality Datapoints for Multiple Novel Intent Detection [24.14668837496296]
  MNID is a framework to detect multiple novel intents with budgeted human annotation cost. It outperforms the baseline methods in terms of accuracy and F1-score.
  arXiv  Detail & Related papers  (2022-05-04T11:32:15Z)
• Continuous representations of intents for dialogue systems [10.031004070657122]
  Up until recently the focus has been on detecting a fixed, discrete, number of seen intents. Recent years have seen some work done on unseen intent detection in the context of zero-shot learning. This paper proposes a novel model where intents are continuous points placed in a specialist Intent Space.
  arXiv  Detail & Related papers  (2021-05-08T15:08:20Z)
• Discriminative Nearest Neighbor Few-Shot Intent Detection by Transferring Natural Language Inference [150.07326223077405]
  Few-shot learning is attracting much attention to mitigate data scarcity. We present a discriminative nearest neighbor classification with deep self-attention. We propose to boost the discriminative ability by transferring a natural language inference (NLI) model.
  arXiv  Detail & Related papers  (2020-10-25T00:39:32Z)
• Learning with Weak Supervision for Email Intent Detection [56.71599262462638]
  We propose to leverage user actions as a source of weak supervision to detect intents in emails. We develop an end-to-end robust deep neural network model for email intent identification.
  arXiv  Detail & Related papers  (2020-05-26T23:41:05Z)
• Intent Mining from past conversations for conversational agent [1.9754522186574608]
  Bots are increasingly being deployed to provide round-the-clock support and to increase customer engagement. Many of the commercial bot building frameworks follow a standard approach that requires one to build and train an intent model to recognize a user input. We have introduced a novel density-based clustering algorithm ITERDB-LabelSCAN for unbalanced data clustering.
  arXiv  Detail & Related papers  (2020-05-22T05:29:13Z)
• AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling [69.59096090788125]
  In this paper, we propose an Adaptive Graph-Interactive Framework (AGIF) for joint multiple intent detection and slot filling. We introduce an intent-slot graph interaction layer to model the strong correlation between the slot and intents. Such an interaction layer is applied to each token adaptively, which has the advantage to automatically extract the relevant intents information.
  arXiv  Detail & Related papers  (2020-04-21T15:07:34Z)
• Efficient Intent Detection with Dual Sentence Encoders [53.16532285820849]
  We introduce intent detection methods backed by pretrained dual sentence encoders such as USE and ConveRT. We demonstrate the usefulness and wide applicability of the proposed intent detectors, showing that they outperform intent detectors based on fine-tuning the full BERT-Large model. We release our code, as well as a new challenging single-domain intent detection dataset.
  arXiv  Detail & Related papers  (2020-03-10T15:33:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.