IDAS: Intent Discovery with Abstractive Summarization
- URL: http://arxiv.org/abs/2305.19783v1
- Date: Wed, 31 May 2023 12:19:40 GMT
- Title: IDAS: Intent Discovery with Abstractive Summarization
- Authors: Maarten De Raedt, Fr\'ederic Godin, Thomas Demeester, Chris Develder
- Abstract summary: We show that recent competitive methods in intent discovery can be outperformed by clustering utterances based on abstractive summaries.
We contribute the IDAS approach, which collects a set of descriptive utterance labels by prompting a Large Language Model.
The utterances and their resulting noisy labels are then encoded by a frozen pre-trained encoder, and subsequently clustered to recover the latent intents.
- Score: 16.731183915325584
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Intent discovery is the task of inferring latent intents from a set of
unlabeled utterances, and is a useful step towards the efficient creation of
new conversational agents. We show that recent competitive methods in intent
discovery can be outperformed by clustering utterances based on abstractive
summaries, i.e., "labels", that retain the core elements while removing
non-essential information. We contribute the IDAS approach, which collects a
set of descriptive utterance labels by prompting a Large Language Model,
starting from a well-chosen seed set of prototypical utterances, to bootstrap
an In-Context Learning procedure to generate labels for non-prototypical
utterances. The utterances and their resulting noisy labels are then encoded by
a frozen pre-trained encoder, and subsequently clustered to recover the latent
intents. For the unsupervised task (without any intent labels) IDAS outperforms
the state-of-the-art by up to +7.42% in standard cluster metrics for the
Banking, StackOverflow, and Transport datasets. For the semi-supervised task
(with labels for a subset of intents) IDAS surpasses 2 recent methods on the
CLINC benchmark without even using labeled data.
Related papers
- Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets [51.74296438621836]
We introduce Scribbles for All, a label and training data generation algorithm for semantic segmentation trained on scribble labels.
The main limitation of scribbles as source for weak supervision is the lack of challenging datasets for scribble segmentation.
Scribbles for All provides scribble labels for several popular segmentation datasets and provides an algorithm to automatically generate scribble labels for any dataset with dense annotations.
arXiv Detail & Related papers (2024-08-22T15:29:08Z) - IntenDD: A Unified Contrastive Learning Approach for Intent Detection
and Discovery [12.905097743551774]
We propose IntenDD, a unified approach leveraging a shared utterance encoding backbone.
IntenDD uses an entirely unsupervised contrastive learning strategy for representation learning.
We find that our approach consistently outperforms competitive baselines across all three tasks.
arXiv Detail & Related papers (2023-10-25T16:50:24Z) - Soft Curriculum for Learning Conditional GANs with Noisy-Labeled and
Uncurated Unlabeled Data [70.25049762295193]
We introduce a novel conditional image generation framework that accepts noisy-labeled and uncurated data during training.
We propose soft curriculum learning, which assigns instance-wise weights for adversarial training while assigning new labels for unlabeled data.
Our experiments show that our approach outperforms existing semi-supervised and label-noise robust methods in terms of both quantitative and qualitative performance.
arXiv Detail & Related papers (2023-07-17T08:31:59Z) - Exploring Structured Semantic Prior for Multi Label Recognition with
Incomplete Labels [60.675714333081466]
Multi-label recognition (MLR) with incomplete labels is very challenging.
Recent works strive to explore the image-to-label correspondence in the vision-language model, ie, CLIP, to compensate for insufficient annotations.
We advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior.
arXiv Detail & Related papers (2023-03-23T12:39:20Z) - New Intent Discovery with Pre-training and Contrastive Learning [21.25371293641141]
New intent discovery aims to uncover novel intent categories from user utterances to expand the set of supported intent classes.
Existing approaches typically rely on a large amount of labeled utterances.
We propose a new contrastive loss to exploit self-supervisory signals in unlabeled data for clustering.
arXiv Detail & Related papers (2022-05-25T17:07:25Z) - Learning with Noisy Labels by Targeted Relabeling [52.0329205268734]
Crowdsourcing platforms are often used to collect datasets for training deep neural networks.
We propose an approach which reserves a fraction of annotations to explicitly relabel highly probable labeling errors.
arXiv Detail & Related papers (2021-10-15T20:37:29Z) - Truth Discovery in Sequence Labels from Crowds [12.181422057560201]
Crowdsourcing platforms, such as Amazon Mechanical Turk (AMT), have been deployed to assist in this purpose.
Existing literature in annotation aggregation assumes that annotations are independent and thus faces challenges when handling the sequential label aggregation tasks.
We propose an optimization-based method that infers the ground truth labels using annotations provided by workers for sequential labeling tasks.
arXiv Detail & Related papers (2021-09-09T19:12:13Z) - Refining Pseudo Labels with Clustering Consensus over Generations for
Unsupervised Object Re-identification [84.72303377833732]
Unsupervised object re-identification targets at learning discriminative representations for object retrieval without any annotations.
We propose to estimate pseudo label similarities between consecutive training generations with clustering consensus and refine pseudo labels with temporally propagated and ensembled pseudo labels.
The proposed pseudo label refinery strategy is simple yet effective and can be seamlessly integrated into existing clustering-based unsupervised re-identification methods.
arXiv Detail & Related papers (2021-06-11T02:42:42Z) - Open Intent Discovery through Unsupervised Semantic Clustering and
Dependency Parsing [44.99113692679489]
This paper proposes an unsupervised two-stage approach to discover intents and generate intent labels automatically from a collection of unlabeled utterances.
We empirically show that the proposed unsupervised approach can generate meaningful intent labels automatically and achieves high precision and recall in utterance clustering and intent discovery.
arXiv Detail & Related papers (2021-04-25T09:36:23Z) - Semi-supervised Relation Extraction via Incremental Meta Self-Training [56.633441255756075]
Semi-Supervised Relation Extraction methods aim to leverage unlabeled data in addition to learning from limited samples.
Existing self-training methods suffer from the gradual drift problem, where noisy pseudo labels on unlabeled data are incorporated during training.
We propose a method called MetaSRE, where a Relation Label Generation Network generates quality assessment on pseudo labels by (meta) learning from the successful and failed attempts on Relation Classification Network as an additional meta-objective.
arXiv Detail & Related papers (2020-10-06T03:54:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.