Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts

• URL: http://arxiv.org/abs/2305.03237v2
• Date: Fri, 23 Feb 2024 09:13:30 GMT
• Title: Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contexts
• Authors: Hao Lang, Yinhe Zheng, Binyuan Hui, Fei Huang, Yongbin Li
• Abstract summary: We introduce a context-aware OOD intent detection (Caro) framework to model multi-turn contexts in OOD intent detection tasks. Caro establishes state-of-the-art performances on multi-turn OOD detection tasks by improving the F1-OOD score of over $29%$ compared to the previous best method.
• Score: 91.43701971416213
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Out-of-Domain (OOD) intent detection is vital for practical dialogue systems, and it usually requires considering multi-turn dialogue contexts. However, most previous OOD intent detection approaches are limited to single dialogue turns. In this paper, we introduce a context-aware OOD intent detection (Caro) framework to model multi-turn contexts in OOD intent detection tasks. Specifically, we follow the information bottleneck principle to extract robust representations from multi-turn dialogue contexts. Two different views are constructed for each input sample and the superfluous information not related to intent detection is removed using a multi-view information bottleneck loss. Moreover, we also explore utilizing unlabeled data in Caro. A two-stage training process is introduced to mine OOD samples from these unlabeled data, and these OOD samples are used to train the resulting model with a bootstrapping approach. Comprehensive experiments demonstrate that Caro establishes state-of-the-art performances on multi-turn OOD detection tasks by improving the F1-OOD score of over $29\%$ compared to the previous best method.

Related papers

• 'No' Matters: Out-of-Distribution Detection in Multimodality Long Dialogue [3.971267935825097]
  This paper aims to improve the user experience that involves multi-round long dialogues by efficiently detecting OOD dialogues and images. We introduce a novel scoring framework named Dialogue Image Aligning and Enhancing Framework (DIAEF) that integrates the visual language models with the novel proposed scores.
  arXiv  Detail & Related papers  (2024-10-31T12:45:54Z)
• Diversity-grounded Channel Prototypical Learning for Out-of-Distribution Intent Detection [18.275098909064127]
  This study presents a novel fine-tuning framework for large language models (LLMs) We construct semantic prototypes for each ID class using a diversity-grounded prompt tuning approach. For a thorough assessment, we benchmark our method against the prevalent fine-tuning approaches.
  arXiv  Detail & Related papers  (2024-09-17T12:07:17Z)
• TagOOD: A Novel Approach to Out-of-Distribution Detection via Vision-Language Representations and Class Center Learning [26.446233594630087]
  We propose textbfTagOOD, a novel approach for OOD detection using vision-language representations. TagOOD trains a lightweight network on the extracted object features to learn representative class centers. These centers capture the central tendencies of IND object classes, minimizing the influence of irrelevant image features during OOD detection.
  arXiv  Detail & Related papers  (2024-08-28T06:37:59Z)
• MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalities [11.884004583641325]
  We introduce the first-of-its-kind benchmark, MultiOOD, characterized by diverse dataset sizes and varying modality combinations. We first evaluate existing unimodal OOD detection algorithms on MultiOOD, observing that the mere inclusion of additional modalities yields substantial improvements. We introduce a novel outlier synthesis method, NP-Mix, which explores broader feature spaces by leveraging the information from nearest neighbor classes.
  arXiv  Detail & Related papers  (2024-05-27T17:59:02Z)
• Negative Label Guided OOD Detection with Pretrained Vision-Language Models [96.67087734472912]
  Out-of-distribution (OOD) detection aims at identifying samples from unknown classes. We propose a novel post hoc OOD detection method, called NegLabel, which takes a vast number of negative labels from extensive corpus databases.
  arXiv  Detail & Related papers  (2024-03-29T09:19:52Z)
• General-Purpose Multi-Modal OOD Detection Framework [5.287829685181842]
  Out-of-distribution (OOD) detection identifies test samples that differ from the training data, which is critical to ensuring the safety and reliability of machine learning (ML) systems. We propose a general-purpose weakly-supervised OOD detection framework, called WOOD, that combines a binary classifier and a contrastive learning component. We evaluate the proposed WOOD model on multiple real-world datasets, and the experimental results demonstrate that the WOOD model outperforms the state-of-the-art methods for multi-modal OOD detection.
  arXiv  Detail & Related papers  (2023-07-24T18:50:49Z)
• SR-OOD: Out-of-Distribution Detection via Sample Repairing [48.272537939227206]
  Out-of-distribution (OOD) detection is a crucial task for ensuring the reliability and robustness of machine learning models. Recent works have shown that generative models often assign high confidence scores to OOD samples, indicating that they fail to capture the semantic information of the data. We take advantage of sample repairing and propose a novel OOD detection framework, namely SR-OOD. Our framework achieves superior performance over the state-of-the-art generative methods in OOD detection.
  arXiv  Detail & Related papers  (2023-05-26T16:35:20Z)
• Pseudo-OOD training for robust language models [78.15712542481859]
  OOD detection is a key component of a reliable machine-learning model for any industry-scale application. We propose POORE - POsthoc pseudo-Ood REgularization, that generates pseudo-OOD samples using in-distribution (IND) data. We extensively evaluate our framework on three real-world dialogue systems, achieving new state-of-the-art in OOD detection.
  arXiv  Detail & Related papers  (2022-10-17T14:32:02Z)
• Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [76.2621758731288]
  We tackle the detection of out-of-distribution (OOD) objects in semantic segmentation. Our main contribution is a new OOD detection architecture called ObsNet associated with a dedicated training scheme based on Local Adversarial Attacks (LAA) We show it obtains top performances both in speed and accuracy when compared to ten recent methods of the literature on three different datasets.
  arXiv  Detail & Related papers  (2021-08-03T17:09:56Z)
• Probing Task-Oriented Dialogue Representation from Language Models [106.02947285212132]
  This paper investigates pre-trained language models to find out which model intrinsically carries the most informative representation for task-oriented dialogue tasks. We fine-tune a feed-forward layer as the classifier probe on top of a fixed pre-trained language model with annotated labels in a supervised way.
  arXiv  Detail & Related papers  (2020-10-26T21:34:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.