Energy-based Unknown Intent Detection with Data Manipulation

• URL: http://arxiv.org/abs/2107.12542v1
• Date: Tue, 27 Jul 2021 01:32:23 GMT
• Title: Energy-based Unknown Intent Detection with Data Manipulation
• Authors: Yawen Ouyang, Jiasheng Ye, Yu Chen, Xinyu Dai, Shujian Huang, Jiajun Chen
• Abstract summary: Unknown intent detection aims to identify the out-of-distribution utterance whose intent has never appeared in the training set. We propose using energy scores for this task as the energy score is theoretically aligned with the density of the input.
• Score: 48.76465739088016
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unknown intent detection aims to identify the out-of-distribution (OOD) utterance whose intent has never appeared in the training set. In this paper, we propose using energy scores for this task as the energy score is theoretically aligned with the density of the input and can be derived from any classifier. However, high-quality OOD utterances are required during the training stage in order to shape the energy gap between OOD and in-distribution (IND), and these utterances are difficult to collect in practice. To tackle this problem, we propose a data manipulation framework to Generate high-quality OOD utterances with importance weighTs (GOT). Experimental results show that the energy-based detector fine-tuned by GOT can achieve state-of-the-art results on two benchmark datasets.

Related papers

• Exploring Energy-Based Models for Out-of-Distribution Detection in Dialect Identification [11.437264025117083]
  This study introduces a novel margin-enhanced joint energy model (MEJEM) tailored specifically for OOD detection in dialects. By integrating a generative model and the energy margin loss, our approach aims to enhance the robustness of dialect identification systems.
  arXiv  Detail & Related papers  (2024-06-26T04:52:48Z)
• Multi-Label Out-of-Distribution Detection with Spectral Normalized Joint Energy [14.149428145967939]
  We introduce Spectral Normalized Joint Energy (SNoJoE), a method that consolidates label-specific information across multiple labels. Our findings indicate that the application of spectral normalization to joint energy scores notably amplifies the model's capability for OOD detection.
  arXiv  Detail & Related papers  (2024-05-08T02:05:38Z)
• Using Semantic Information for Defining and Detecting OOD Inputs [3.9577682622066264]
  Out-of-distribution (OOD) detection has received some attention recently. We demonstrate that the current detectors inherit the biases in the training dataset. This can render the current OOD detectors impermeable to inputs lying outside the training distribution but with the same semantic information. We perform OOD detection on semantic information extracted from the training data of MNIST and COCO datasets.
  arXiv  Detail & Related papers  (2023-02-21T21:31:20Z)
• Unsupervised Evaluation of Out-of-distribution Detection: A Data-centric Perspective [55.45202687256175]
  Out-of-distribution (OOD) detection methods assume that they have test ground truths, i.e., whether individual test samples are in-distribution (IND) or OOD. In this paper, we are the first to introduce the unsupervised evaluation problem in OOD detection. We propose three methods to compute Gscore as an unsupervised indicator of OOD detection performance.
  arXiv  Detail & Related papers  (2023-02-16T13:34:35Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)
• Master's Thesis: Out-of-distribution Detection with Energy-based Models [0.0]
  Deep learning is increasingly applied in security-critical situations such as autonomous driving and medical diagnosis. Researchers recently found that neural networks are overly confident in their predictions, even on data they have never seen before. In this thesis, we investigate the capabilities of EBMs at the task of fitting the training data distribution to perform detection of out-of-distribution (OOD) inputs.
  arXiv  Detail & Related papers  (2023-01-28T15:40:30Z)
• GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection [67.90365841083951]
  We develop a new graph contrastive learning framework GOOD-D for detecting OOD graphs without using any ground-truth labels. GOOD-D is able to capture the latent ID patterns and accurately detect OOD graphs based on the semantic inconsistency in different granularities. As a pioneering work in unsupervised graph-level OOD detection, we build a comprehensive benchmark to compare our proposed approach with different state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-11-08T12:41:58Z)
• Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning [40.96874034407684]
  We propose a simple but strong energy-based score function to detect OOD where the energy scores of OOD samples are higher than IND samples. Comprehensive experiments and analysis prove our method helps disentangle confidence score distributions of IND and OOD data.
  arXiv  Detail & Related papers  (2022-10-17T08:19:01Z)
• Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities [104.02531442035483]
  The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods. We show that binary discrimination between in- and (different) out-distributions is equivalent to several distinct formulations of the OOD detection problem. We also show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function.
  arXiv  Detail & Related papers  (2022-06-20T16:32:49Z)
• Provably Robust Detection of Out-of-distribution Data (almost) for free [124.14121487542613]
  Deep neural networks are known to produce highly overconfident predictions on out-of-distribution (OOD) data. In this paper we propose a novel method where from first principles we combine a certifiable OOD detector with a standard classifier into an OOD aware classifier. In this way we achieve the best of two worlds: certifiably adversarially robust OOD detection, even for OOD samples close to the in-distribution, without loss in prediction accuracy and close to state-of-the-art OOD detection performance for non-manipulated OOD data.
  arXiv  Detail & Related papers  (2021-06-08T11:40:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.