Estimating Soft Labels for Out-of-Domain Intent Detection

• URL: http://arxiv.org/abs/2211.05561v1
• Date: Thu, 10 Nov 2022 13:31:13 GMT
• Title: Estimating Soft Labels for Out-of-Domain Intent Detection
• Authors: Hao Lang, Yinhe Zheng, Jian Sun, Fei Huang, Luo Si, Yongbin Li
• Abstract summary: Out-of-Domain (OOD) intent detection is important for practical dialog systems. We propose an adaptive soft pseudo labeling (ASoul) method that can estimate soft labels for pseudo OOD samples.
• Score: 122.68266151023676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Out-of-Domain (OOD) intent detection is important for practical dialog systems. To alleviate the issue of lacking OOD training samples, some works propose synthesizing pseudo OOD samples and directly assigning one-hot OOD labels to these pseudo samples. However, these one-hot labels introduce noises to the training process because some hard pseudo OOD samples may coincide with In-Domain (IND) intents. In this paper, we propose an adaptive soft pseudo labeling (ASoul) method that can estimate soft labels for pseudo OOD samples when training OOD detectors. Semantic connections between pseudo OOD samples and IND intents are captured using an embedding graph. A co-training framework is further introduced to produce resulting soft labels following the smoothness assumption, i.e., close samples are likely to have similar labels. Extensive experiments on three benchmark datasets show that ASoul consistently improves the OOD detection performance and outperforms various competitive baselines.

Related papers

• COOD: Concept-based Zero-shot OOD Detection [12.361461338978732]
  We introduce COOD, a novel zero-shot multi-label OOD detection framework. By enriching the semantic space with both positive and negative concepts for each label, our approach models complex label dependencies. Our method significantly outperforms existing approaches, achieving approximately 95% average AUROC on both VOC and datasets.
  arXiv  Detail & Related papers  (2024-11-15T08:15:48Z)
• Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox [70.57120710151105]
  Most existing out-of-distribution (OOD) detection benchmarks classify samples with novel labels as the OOD data. Some marginal OOD samples actually have close semantic contents to the in-distribution (ID) sample, which makes determining the OOD sample a Sorites Paradox. We construct a benchmark named Incremental Shift OOD (IS-OOD) to address the issue.
  arXiv  Detail & Related papers  (2024-06-14T09:27:56Z)
• Negative Label Guided OOD Detection with Pretrained Vision-Language Models [96.67087734472912]
  Out-of-distribution (OOD) detection aims at identifying samples from unknown classes. We propose a novel post hoc OOD detection method, called NegLabel, which takes a vast number of negative labels from extensive corpus databases.
  arXiv  Detail & Related papers  (2024-03-29T09:19:52Z)
• APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection [40.846633965439956]
  This paper focuses on a few-shot OOD setting where there are only a few labeled IND data and massive unlabeled mixed data. We propose an adaptive pseudo-labeling (APP) method for few-shot OOD detection.
  arXiv  Detail & Related papers  (2023-10-20T09:48:52Z)
• Pseudo Outlier Exposure for Out-of-Distribution Detection using Pretrained Transformers [3.8839179829686126]
  A rejection network can be trained with ID and diverse outlier samples to detect test OOD samples. We propose a method called Pseudo Outlier Exposure (POE) that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes. Our method does not require any external OOD data and can be easily implemented within off-the-shelf Transformers.
  arXiv  Detail & Related papers  (2023-07-18T17:29:23Z)
• Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection [72.35532598131176]
  We propose an unsupervised method to detect OOD samples using a $k$-NN density estimate. We leverage a recent insight about label smoothing, which we call the emphLabel Smoothed Embedding Hypothesis We show that our proposal outperforms many OOD baselines and also provide new finite-sample high-probability statistical results.
  arXiv  Detail & Related papers  (2021-02-09T21:04:44Z)
• On The Consistency Training for Open-Set Semi-Supervised Learning [44.046578996049654]
  We study how OOD samples affect training in both low- and high-dimensional spaces. Our method makes better use of OOD samples and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2021-01-19T12:38:17Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)
• Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning [54.85397562961903]
  Semi-supervised learning (SSL) has been proposed to leverage unlabeled data for training powerful models when only limited labeled data is available. We address a more complex novel scenario named open-set SSL, where out-of-distribution (OOD) samples are contained in unlabeled data. Our method achieves state-of-the-art results by successfully eliminating the effect of OOD samples.
  arXiv  Detail & Related papers  (2020-07-22T10:33:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.