Unsupervised Domain Adaptation: A Reality Check

• URL: http://arxiv.org/abs/2111.15672v1
• Date: Tue, 30 Nov 2021 18:59:04 GMT
• Title: Unsupervised Domain Adaptation: A Reality Check
• Authors: Kevin Musgrave, Serge Belongie, Ser-Nam Lim
• Abstract summary: We show via large-scale experimentation that the difference in accuracy between UDA algorithms is smaller than previously thought. This is despite the fact that validation methods are a crucial component of any UDA train/val pipeline.
• Score: 23.79809492395849
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Interest in unsupervised domain adaptation (UDA) has surged in recent years, resulting in a plethora of new algorithms. However, as is often the case in fast-moving fields, baseline algorithms are not tested to the extent that they should be. Furthermore, little attention has been paid to validation methods, i.e. the methods for estimating the accuracy of a model in the absence of target domain labels. This is despite the fact that validation methods are a crucial component of any UDA train/val pipeline. In this paper, we show via large-scale experimentation that 1) in the oracle setting, the difference in accuracy between UDA algorithms is smaller than previously thought, 2) state-of-the-art validation methods are not well-correlated with accuracy, and 3) differences between UDA algorithms are dwarfed by the drop in accuracy caused by validation methods.

Related papers

• Simulations of Common Unsupervised Domain Adaptation Algorithms for Image Classification [14.98782101639076]
  Domain adaptation (DA) is a machine learning technique that aims to address this problem by reducing the differences between domains. This paper presents simulation-based algorithms of recent DA techniques, mainly related to unsupervised domain adaptation (UDA) Our study compares these techniques with public data sets and diverse characteristics, highlighting their respective strengths and drawbacks.
  arXiv  Detail & Related papers  (2025-02-15T06:58:57Z)
• An Efficient Outlier Detection Algorithm for Data Streaming [51.56874851156008]
  Traditional outlier detection methods, such as the Local Outlier Factor (LOF) algorithm, struggle with real-time data. We propose a novel approach to enhance the efficiency of LOF algorithms for online anomaly detection, named the Efficient Incremental LOF (EILOF) algorithm. The EILOF algorithm not only significantly reduces computational costs, but also systematically improves detection accuracy when the number of additional points increases.
  arXiv  Detail & Related papers  (2025-01-02T05:12:43Z)
• Unsupervised Domain Adaptation Via Data Pruning [0.0]
  We consider the problem from the perspective of unsupervised domain adaptation (UDA) We propose AdaPrune, a method for UDA whereby training examples are removed to attempt to align the training distribution to that of the target data. As a method for UDA, we show that AdaPrune outperforms related techniques, and is complementary to other UDA algorithms such as CORAL.
  arXiv  Detail & Related papers  (2024-09-18T15:48:59Z)
• Better Practices for Domain Adaptation [62.70267990659201]
  Domain adaptation (DA) aims to provide frameworks for adapting models to deployment data without using labels. Unclear validation protocol for DA has led to bad practices in the literature. We show challenges across all three branches of domain adaptation methodology.
  arXiv  Detail & Related papers  (2023-09-07T17:44:18Z)
• OpenOOD: Benchmarking Generalized Out-of-Distribution Detection [60.13300701826931]
  Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications. The field currently lacks a unified, strictly formulated, and comprehensive benchmark. We build a unified, well-structured called OpenOOD, which implements over 30 methods developed in relevant fields.
  arXiv  Detail & Related papers  (2022-10-13T17:59:57Z)
• Interpolation-based Contrastive Learning for Few-Label Semi-Supervised Learning [43.51182049644767]
  Semi-supervised learning (SSL) has long been proved to be an effective technique to construct powerful models with limited labels. Regularization-based methods which force the perturbed samples to have similar predictions with the original ones have attracted much attention. We propose a novel contrastive loss to guide the embedding of the learned network to change linearly between samples.
  arXiv  Detail & Related papers  (2022-02-24T06:00:05Z)
• E-ADDA: Unsupervised Adversarial Domain Adaptation Enhanced by a New Mahalanobis Distance Loss for Smart Computing [25.510639595356597]
  In smart computing, the labels of training samples for a specific task are not always abundant. We propose a novel UDA algorithm, textitE-ADDA, which uses both a novel variation of the Mahalanobis distance loss and an out-of-distribution detection subroutine. In the acoustic modality, E-ADDA outperforms several state-of-the-art UDA algorithms by up to 29.8%, measured in the f1 score. In the computer vision modality, the evaluation results suggest that we achieve new state-of-the-art performance on popular UDA
  arXiv  Detail & Related papers  (2022-01-24T23:20:55Z)
• Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Density [125.64297244986552]
  We propose an unsupervised validation criterion that measures the density of soft neighborhoods by computing the entropy of the similarity distribution between points. Our criterion is simpler than competing validation methods, yet more effective.
  arXiv  Detail & Related papers  (2021-08-24T17:41:45Z)
• Understanding the Limits of Unsupervised Domain Adaptation via Data Poisoning [66.80663779176979]
  Unsupervised domain adaptation (UDA) enables cross-domain learning without target domain labels. We show the insufficiency of minimizing source domain error and marginal distribution mismatch for a guaranteed reduction in the target domain error. Motivated from this, we propose novel data poisoning attacks to fool UDA methods into learning representations that produce large target domain errors.
  arXiv  Detail & Related papers  (2021-07-08T15:51:14Z)
• Unsupervised Out-of-Domain Detection via Pre-trained Transformers [56.689635664358256]
  Out-of-domain inputs can lead to unpredictable outputs and sometimes catastrophic safety issues. Our work tackles the problem of detecting out-of-domain samples with only unsupervised in-domain data. Two domain-specific fine-tuning approaches are further proposed to boost detection accuracy.
  arXiv  Detail & Related papers  (2021-06-02T05:21:25Z)
• Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data [40.82826366059613]
  Unsupervised Data Augmentation (UDA) is a semi-supervised technique that applies a consistency loss to penalize differences between a model's predictions. In this paper, we re-examine UDA and demonstrate its efficacy on several sequential tasks. We find that applying its consistency loss affords meaningful gains without any unlabeled data at all.
  arXiv  Detail & Related papers  (2020-10-22T18:01:51Z)
• Rethinking Distributional Matching Based Domain Adaptation [111.15106414932413]
  Domain adaptation (DA) is a technique that transfers predictive models trained on a labeled source domain to an unlabeled target domain. Most popular DA algorithms are based on distributional matching (DM) In this paper, we first systematically analyze the limitations of DM based methods, and then build new benchmarks with more realistic domain shifts.
  arXiv  Detail & Related papers  (2020-06-23T21:55:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.