Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection

• URL: http://arxiv.org/abs/2101.03778v1
• Date: Mon, 11 Jan 2021 09:10:58 GMT
• Title: Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection
• Authors: Alexander Podolskiy and Dmitry Lipin and Andrey Bout and Ekaterina Artemova and Irina Piontkovskaya
• Abstract summary: This paper conducts a thorough comparison of out-of-domain intent detection methods. We evaluate multiple contextual encoders and methods, proven to be efficient, on three standard datasets for intent classification. Our main findings show that fine-tuning Transformer-based encoders on in-domain data leads to superior results.
• Score: 60.88952532574564
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Real-life applications, heavily relying on machine learning, such as dialog systems, demand out-of-domain detection methods. Intent classification models should be equipped with a mechanism to distinguish seen intents from unseen ones so that the dialog agent is capable of rejecting the latter and avoiding undesired behavior. However, despite increasing attention paid to the task, the best practices for out-of-domain intent detection have not yet been fully established. This paper conducts a thorough comparison of out-of-domain intent detection methods. We prioritize the methods, not requiring access to out-of-domain data during training, gathering of which is extremely time- and labor-consuming due to lexical and stylistic variation of user utterances. We evaluate multiple contextual encoders and methods, proven to be efficient, on three standard datasets for intent classification, expanded with out-of-domain utterances. Our main findings show that fine-tuning Transformer-based encoders on in-domain data leads to superior results. Mahalanobis distance, together with utterance representations, derived from Transformer-based encoders, outperforms other methods by a wide margin and establishes new state-of-the-art results for all datasets. The broader analysis shows that the reason for success lies in the fact that the fine-tuned Transformer is capable of constructing homogeneous representations of in-domain utterances, revealing geometrical disparity to out of domain utterances. In turn, the Mahalanobis distance captures this disparity easily.

Related papers

• Low-confidence Samples Matter for Domain Adaptation [47.552605279925736]
  Domain adaptation (DA) aims to transfer knowledge from a label-rich source domain to a related but label-scarce target domain. We propose a novel contrastive learning method by processing low-confidence samples. We evaluate the proposed method in both unsupervised and semi-supervised DA settings.
  arXiv  Detail & Related papers  (2022-02-06T15:45:45Z)
• Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection [85.11649974840758]
  3D object detection networks tend to be biased towards the data they are trained on. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar-based 3D object detectors.
  arXiv  Detail & Related papers  (2021-11-30T18:42:42Z)
• Conditional Extreme Value Theory for Open Set Video Domain Adaptation [17.474956295874797]
  We propose an open-set video domain adaptation approach to mitigate the domain discrepancy between the source and target data. To alleviate the negative transfer issue, weights computed by the distance from the sample entropy to the threshold are leveraged in adversarial learning. The proposed method has been thoroughly evaluated on both small-scale and large-scale cross-domain video datasets.
  arXiv  Detail & Related papers  (2021-09-01T10:51:50Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)
• Unsupervised Out-of-Domain Detection via Pre-trained Transformers [56.689635664358256]
  Out-of-domain inputs can lead to unpredictable outputs and sometimes catastrophic safety issues. Our work tackles the problem of detecting out-of-domain samples with only unsupervised in-domain data. Two domain-specific fine-tuning approaches are further proposed to boost detection accuracy.
  arXiv  Detail & Related papers  (2021-06-02T05:21:25Z)
• Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision [73.76277367528657]
  Convolutional neural network-based approaches have achieved remarkable progress in semantic segmentation. To cope with this limitation, automatically annotated data generated from graphic engines are used to train segmentation models. We propose a two-step self-supervised domain adaptation approach to minimize the inter-domain and intra-domain gap together.
  arXiv  Detail & Related papers  (2020-04-16T15:24:11Z)
• Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation [62.29076080124199]
  This paper proposes a novel coarse-to-fine feature adaptation approach to cross-domain object detection. At the coarse-grained stage, foreground regions are extracted by adopting the attention mechanism, and aligned according to their marginal distributions. At the fine-grained stage, we conduct conditional distribution alignment of foregrounds by minimizing the distance of global prototypes with the same category but from different domains.
  arXiv  Detail & Related papers  (2020-03-23T13:40:06Z)
• Focus on Semantic Consistency for Cross-domain Crowd Understanding [34.560447389853614]
  Some domain adaptation algorithms try to liberate it by training models with synthetic data. We found that a mass of estimation errors in the background areas impede the performance of the existing methods. In this paper, we propose a domain adaptation method to eliminate it.
  arXiv  Detail & Related papers  (2020-02-20T08:51:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.