Unsupervised Out-of-Domain Detection via Pre-trained Transformers

• URL: http://arxiv.org/abs/2106.00948v1
• Date: Wed, 2 Jun 2021 05:21:25 GMT
• Title: Unsupervised Out-of-Domain Detection via Pre-trained Transformers
• Authors: Keyang Xu, Tongzheng Ren, Shikun Zhang, Yihao Feng and Caiming Xiong
• Abstract summary: Out-of-domain inputs can lead to unpredictable outputs and sometimes catastrophic safety issues. Our work tackles the problem of detecting out-of-domain samples with only unsupervised in-domain data. Two domain-specific fine-tuning approaches are further proposed to boost detection accuracy.
• Score: 56.689635664358256
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deployed real-world machine learning applications are often subject to uncontrolled and even potentially malicious inputs. Those out-of-domain inputs can lead to unpredictable outputs and sometimes catastrophic safety issues. Prior studies on out-of-domain detection require in-domain task labels and are limited to supervised classification scenarios. Our work tackles the problem of detecting out-of-domain samples with only unsupervised in-domain data. We utilize the latent representations of pre-trained transformers and propose a simple yet effective method to transform features across all layers to construct out-of-domain detectors efficiently. Two domain-specific fine-tuning approaches are further proposed to boost detection accuracy. Our empirical evaluations of related methods on two datasets validate that our method greatly improves out-of-domain detection ability in a more general scenario.

Related papers

• Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment [17.485775402656127]
  A base detector can outperform existing methods for single domain generalization by a good margin. We introduce a method to align detections from multiple views, considering both classification and localization outputs. Our approach is detector-agnostic and can be seamlessly applied to both single-stage and two-stage detectors.
  arXiv  Detail & Related papers  (2024-05-23T12:29:25Z)
• DATR: Unsupervised Domain Adaptive Detection Transformer with Dataset-Level Adaptation and Prototypical Alignment [7.768332621617199]
  We introduce a strong DETR-based detector named Domain Adaptive detection TRansformer ( DATR) for unsupervised domain adaptation of object detection. Our proposed DATR incorporates a mean-teacher based self-training framework, utilizing pseudo-labels generated by the teacher model to further mitigate domain bias. Experiments demonstrate superior performance and generalization capabilities of our proposed DATR in multiple domain adaptation scenarios.
  arXiv  Detail & Related papers  (2024-05-20T03:48:45Z)
• Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement [79.2994130944482]
  We design a Domain Disentanglement Faster-RCNN (DDF) to eliminate the source-specific information in the features for detection task learning. Our DDF method facilitates the feature disentanglement at the global and local stages, with a Global Triplet Disentanglement (GTD) module and an Instance Similarity Disentanglement (ISD) module. By outperforming state-of-the-art methods on four benchmark UDA object detection tasks, our DDF method is demonstrated to be effective with wide applicability.
  arXiv  Detail & Related papers  (2022-01-06T05:43:01Z)
• Adversarially Trained Object Detector for Unsupervised Domain Adaptation [4.9631159466100305]
  We demonstrate that adversarial training in the source domain can be employed as a new approach for unsupervised domain adaptation. We propose a method that combines adversarial training and feature alignment to ensure the improved alignment of robust features with the target domain.
  arXiv  Detail & Related papers  (2021-09-13T07:21:28Z)
• Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers [141.70707071815653]
  We propose a novel Sequence Feature Alignment (SFA) method that is specially designed for the adaptation of detection transformers. SFA consists of a domain query-based feature alignment (DQFA) module and a token-wise feature alignment (TDA) module. Experiments on three challenging benchmarks show that SFA outperforms state-of-the-art domain adaptive object detection methods.
  arXiv  Detail & Related papers  (2021-07-27T07:17:12Z)
• Domain Adaptive YOLO for One-Stage Cross-Domain Detection [4.596221278839825]
  Domain Adaptive YOLO (DA-YOLO) is proposed to improve cross-domain performance for one-stage detectors. We evaluate our proposed method on popular datasets like Cityscapes, KITTI, SIM10K and etc.
  arXiv  Detail & Related papers  (2021-06-26T04:17:42Z)
• Uncertainty-Aware Unsupervised Domain Adaptation in Object Detection [34.18382705952121]
  Unlabelled domain adaptive object detection aims to adapt detectors from a labelled source domain to an unsupervised target domain. adversarial learning may impair the alignment of well-aligned samples as it merely aligns the global distributions across domains. We design an uncertainty-aware domain adaptation network (UaDAN) that introduces conditional adversarial learning to align well-aligned and poorly-aligned samples separately.
  arXiv  Detail & Related papers  (2021-02-27T15:04:07Z)
• Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection [60.88952532574564]
  This paper conducts a thorough comparison of out-of-domain intent detection methods. We evaluate multiple contextual encoders and methods, proven to be efficient, on three standard datasets for intent classification. Our main findings show that fine-tuning Transformer-based encoders on in-domain data leads to superior results.
  arXiv  Detail & Related papers  (2021-01-11T09:10:58Z)
• Cross-domain Detection via Graph-induced Prototype Alignment [114.8952035552862]
  We propose a Graph-induced Prototype Alignment (GPA) framework to seek for category-level domain alignment. In addition, in order to alleviate the negative effect of class-imbalance on domain adaptation, we design a Class-reweighted Contrastive Loss. Our approach outperforms existing methods with a remarkable margin.
  arXiv  Detail & Related papers  (2020-03-28T17:46:55Z)
• Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation [62.29076080124199]
  This paper proposes a novel coarse-to-fine feature adaptation approach to cross-domain object detection. At the coarse-grained stage, foreground regions are extracted by adopting the attention mechanism, and aligned according to their marginal distributions. At the fine-grained stage, we conduct conditional distribution alignment of foregrounds by minimizing the distance of global prototypes with the same category but from different domains.
  arXiv  Detail & Related papers  (2020-03-23T13:40:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.