Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning

• URL: http://arxiv.org/abs/2105.14289v1
• Date: Sat, 29 May 2021 12:54:22 GMT
• Title: Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning
• Authors: Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Zijun Liu, Yanan Wu, Hong Xu, Huixing Jiang and Weiran Xu
• Abstract summary: Key challenge of OOD detection is to learn discriminative semantic features. We propose a supervised contrastive learning objective to minimize intra-class variance. We employ an adversarial augmentation mechanism to obtain pseudo diverse views of a sample.
• Score: 16.77134235390429
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. A key challenge of OOD detection is to learn discriminative semantic features. Traditional cross-entropy loss only focuses on whether a sample is correctly classified, and does not explicitly distinguish the margins between categories. In this paper, we propose a supervised contrastive learning objective to minimize intra-class variance by pulling together in-domain intents belonging to the same class and maximize inter-class variance by pushing apart samples from different classes. Besides, we employ an adversarial augmentation mechanism to obtain pseudo diverse views of a sample in the latent space. Experiments on two public datasets prove the effectiveness of our method capturing discriminative representations for OOD detection.

Related papers

• Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection [75.02249869573994]
  In open-set scenarios, the unlabeled dataset contains both in-distribution (ID) classes and out-of-distribution (OOD) classes. Applying semi-supervised detectors in such settings can lead to misclassifying OOD class as ID classes. We propose a simple yet effective method, termed Collaborative Feature-Logits Detector (CFL-Detector)
  arXiv  Detail & Related papers  (2024-11-20T02:57:35Z)
• Diversity-grounded Channel Prototypical Learning for Out-of-Distribution Intent Detection [18.275098909064127]
  This study presents a novel fine-tuning framework for large language models (LLMs) We construct semantic prototypes for each ID class using a diversity-grounded prompt tuning approach. For a thorough assessment, we benchmark our method against the prevalent fine-tuning approaches.
  arXiv  Detail & Related papers  (2024-09-17T12:07:17Z)
• Out-of-Distribution Detection Using Peer-Class Generated by Large Language Model [0.0]
  Out-of-distribution (OOD) detection is a critical task to ensure the reliability and security of machine learning models. In this paper, a novel method called ODPC is proposed, in which specific prompts to generate OOD peer classes of ID semantics are designed by a large language model. Experiments on five benchmark datasets show that the method we propose can yield state-of-the-art results.
  arXiv  Detail & Related papers  (2024-03-20T06:04:05Z)
• WDiscOOD: Out-of-Distribution Detection via Whitened Linear Discriminant Analysis [21.023001428704085]
  We propose a novel feature-space OOD detection score based on class-specific and class-agnostic information. The efficacy of our method, named WDiscOOD, is verified on the large-scale ImageNet-1k benchmark.
  arXiv  Detail & Related papers  (2023-03-14T00:13:57Z)
• Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems [61.11799513362704]
  We propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective.
  arXiv  Detail & Related papers  (2023-03-03T02:07:40Z)
• Cluster-aware Contrastive Learning for Unsupervised Out-of-distribution Detection [0.0]
  Unsupervised out-of-distribution (OOD) Detection aims to separate the samples falling outside the distribution of training data without label information. We propose Cluster-aware Contrastive Learning (CCL) framework for unsupervised OOD detection, which considers both instance-level and semantic-level information.
  arXiv  Detail & Related papers  (2023-02-06T07:21:03Z)
• Understanding the properties and limitations of contrastive learning for Out-of-Distribution detection [3.2689702143620143]
  A popular approach to out-of-distribution (OOD) detection is based on a self-supervised learning technique referred to as contrastive learning. In this paper, we aim to understand the effectiveness and limitation of existing contrastive learning methods for OOD detection.
  arXiv  Detail & Related papers  (2022-11-06T17:33:29Z)
• Towards Intrinsic Common Discriminative Features Learning for Face Forgery Detection using Adversarial Learning [59.548960057358435]
  We propose a novel method which utilizes adversarial learning to eliminate the negative effect of different forgery methods and facial identities. Our face forgery detection model learns to extract common discriminative features through eliminating the effect of forgery methods and facial identities.
  arXiv  Detail & Related papers  (2022-07-08T09:23:59Z)
• Dual Contrastive Learning for General Face Forgery Detection [64.41970626226221]
  We propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which constructs positive and negative paired data. To explore the essential discrepancies, Intra-Instance Contrastive Learning (Intra-ICL) is introduced to focus on the local content inconsistencies prevalent in the forged faces.
  arXiv  Detail & Related papers  (2021-12-27T05:44:40Z)
• A Low Rank Promoting Prior for Unsupervised Contrastive Learning [108.91406719395417]
  We construct a novel probabilistic graphical model that effectively incorporates the low rank promoting prior into the framework of contrastive learning. Our hypothesis explicitly requires that all the samples belonging to the same instance class lie on the same subspace with small dimension. Empirical evidences show that the proposed algorithm clearly surpasses the state-of-the-art approaches on multiple benchmarks.
  arXiv  Detail & Related papers  (2021-08-05T15:58:25Z)
• Category Contrast for Unsupervised Domain Adaptation in Visual Tasks [92.9990560760593]
  We propose a novel Category Contrast technique (CaCo) that introduces semantic priors on top of instance discrimination for visual UDA tasks. CaCo is complementary to existing UDA methods and generalizable to other learning setups such as semi-supervised learning, unsupervised model adaptation, etc.
  arXiv  Detail & Related papers  (2021-06-05T12:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.