Discovering New Intents Using Latent Variables
- URL: http://arxiv.org/abs/2210.11804v1
- Date: Fri, 21 Oct 2022 08:29:45 GMT
- Title: Discovering New Intents Using Latent Variables
- Authors: Yunhua Zhou, Peiju Liu, Yuxin Wang, Xipeng QIu
- Abstract summary: We propose a probabilistic framework for discovering intents where intent assignments are treated as latent variables.
In E-step, we conduct discovering intents and explore the intrinsic structure of unlabeled data by the posterior of intent assignments.
In M-step, we alleviate the forgetting of prior knowledge transferred from known intents by optimizing the discrimination of labeled data.
- Score: 51.50374666602328
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Discovering new intents is of great significance to establishing Bootstrapped
Task-Oriented Dialogue System. Most existing methods either lack the ability to
transfer prior knowledge in the known intent data or fall into the dilemma of
forgetting prior knowledge in the follow-up. More importantly, these methods do
not deeply explore the intrinsic structure of unlabeled data, so they can not
seek out the characteristics that make an intent in general. In this paper,
starting from the intuition that discovering intents could be beneficial to the
identification of the known intents, we propose a probabilistic framework for
discovering intents where intent assignments are treated as latent variables.
We adopt Expectation Maximization framework for optimization. Specifically, In
E-step, we conduct discovering intents and explore the intrinsic structure of
unlabeled data by the posterior of intent assignments. In M-step, we alleviate
the forgetting of prior knowledge transferred from known intents by optimizing
the discrimination of labeled data. Extensive experiments conducted in three
challenging real-world datasets demonstrate our method can achieve substantial
improvements.
Related papers
- ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous
Driving [96.92499034935466]
End-to-end differentiable learning for autonomous driving has recently become a prominent paradigm.
One main bottleneck lies in its voracious appetite for high-quality labeled data.
We propose a planning-oriented active learning method which progressively annotates part of collected raw data.
arXiv Detail & Related papers (2024-03-05T11:39:07Z) - XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification.
XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations.
Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
arXiv Detail & Related papers (2023-10-09T08:07:04Z) - Knowledge Combination to Learn Rotated Detection Without Rotated
Annotation [53.439096583978504]
Rotated bounding boxes drastically reduce output ambiguity of elongated objects.
Despite the effectiveness, rotated detectors are not widely employed.
We propose a framework that allows the model to predict precise rotated boxes.
arXiv Detail & Related papers (2023-04-05T03:07:36Z) - New Intent Discovery with Pre-training and Contrastive Learning [21.25371293641141]
New intent discovery aims to uncover novel intent categories from user utterances to expand the set of supported intent classes.
Existing approaches typically rely on a large amount of labeled utterances.
We propose a new contrastive loss to exploit self-supervisory signals in unlabeled data for clustering.
arXiv Detail & Related papers (2022-05-25T17:07:25Z) - Learning Discriminative Representations and Decision Boundaries for Open
Intent Detection [16.10123071366136]
Open intent detection is a significant problem in natural language understanding.
We propose DA-ADB, which learns distance-aware intent representations and adaptive decision boundaries for open intent detection.
Our framework achieves substantial improvements on three benchmark datasets.
arXiv Detail & Related papers (2022-03-11T10:02:09Z) - A Bayesian Framework for Information-Theoretic Probing [51.98576673620385]
We argue that probing should be seen as approximating a mutual information.
This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences.
This paper proposes a new framework to measure what we term Bayesian mutual information.
arXiv Detail & Related papers (2021-09-08T18:08:36Z) - Generalized Zero-shot Intent Detection via Commonsense Knowledge [5.398580049917152]
We propose RIDE: an intent detection model that leverages commonsense knowledge in an unsupervised fashion to overcome the issue of training data scarcity.
RIDE computes robust and generalizable relationship meta-features that capture deep semantic relationships between utterances and intent labels.
Our extensive experimental analysis on three widely-used intent detection benchmarks shows that relationship meta-features significantly increase the accuracy of detecting both seen and unseen intents.
arXiv Detail & Related papers (2021-02-04T23:36:41Z) - Deep Open Intent Classification with Adaptive Decision Boundary [21.478553057876972]
We propose a post-processing method to learn the adaptive decision boundary (ADB) for open intent classification.
Specifically, we propose a new loss function to balance both the empirical risk and the open space risk.
Our approach is surprisingly insensitive with less labeled data and fewer known intents.
arXiv Detail & Related papers (2020-12-18T13:05:11Z) - Discovering New Intents with Deep Aligned Clustering [19.11073686645496]
We propose an effective method, Deep Aligned Clustering, to discover new intents with the aid of limited known intent data.
With an unknown number of new intents, we predict the number of intent categories by eliminating low-confidence intent-wise clusters.
Experiments on two benchmark datasets show that our method is more robust and achieves substantial improvements over the state-of-the-art methods.
arXiv Detail & Related papers (2020-12-16T14:32:06Z) - Incremental Object Detection via Meta-Learning [77.55310507917012]
We propose a meta-learning approach that learns to reshape model gradients, such that information across incremental tasks is optimally shared.
In comparison to existing meta-learning methods, our approach is task-agnostic, allows incremental addition of new-classes and scales to high-capacity models for object detection.
arXiv Detail & Related papers (2020-03-17T13:40:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.