Free Lunch for Domain Adversarial Training: Environment Label Smoothing

• URL: http://arxiv.org/abs/2302.00194v1
• Date: Wed, 1 Feb 2023 02:55:26 GMT
• Title: Free Lunch for Domain Adversarial Training: Environment Label Smoothing
• Authors: YiFan Zhang, Xue Wang, Jian Liang, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan
• Abstract summary: We propose Environment Label Smoothing (ELS) to improve training stability, local convergence, and robustness to noisy environment labels. We yield state-of-art results on a wide range of domain generalization/adaptation tasks, particularly when the environment labels are highly noisy.
• Score: 82.85757548355566
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A fundamental challenge for machine learning models is how to generalize learned models for out-of-distribution (OOD) data. Among various approaches, exploiting invariant features by Domain Adversarial Training (DAT) received widespread attention. Despite its success, we observe training instability from DAT, mostly due to over-confident domain discriminator and environment label noise. To address this issue, we proposed Environment Label Smoothing (ELS), which encourages the discriminator to output soft probability, which thus reduces the confidence of the discriminator and alleviates the impact of noisy environment labels. We demonstrate, both experimentally and theoretically, that ELS can improve training stability, local convergence, and robustness to noisy environment labels. By incorporating ELS with DAT methods, we are able to yield state-of-art results on a wide range of domain generalization/adaptation tasks, particularly when the environment labels are highly noisy.

Related papers

• Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selection [82.43311784594384]
  Real-world datasets contain not only noisy labels but also class imbalance. We propose a simple yet effective method to address noisy labels in imbalanced datasets.
  arXiv  Detail & Related papers  (2024-02-17T10:34:53Z)
• Learning Cautiously in Federated Learning with Noisy and Heterogeneous Clients [4.782145666637457]
  Federated learning (FL) is a distributed framework for collaboratively training with privacy guarantees. In real-world scenarios, clients may have Non-IID data (local class imbalance) with poor annotation quality (label noise) We propose FedCNI without using an additional clean proxy dataset. It includes a noise-resilient local solver and a robust global aggregator.
  arXiv  Detail & Related papers  (2023-04-06T06:47:14Z)
• Adapting to Online Label Shift with Provable Guarantees [137.89382409682233]
  We formulate and investigate the problem of online label shift. The non-stationarity and lack of supervision make the problem challenging to be tackled. Our algorithms enjoy optimal dynamic regret, indicating that performance is competitive with a clairvoyant nature.
  arXiv  Detail & Related papers  (2022-07-05T15:43:14Z)
• FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels [49.47228898303909]
  Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. We develop a simple two-level sampling method "FedNoiL" that selects clients for more robust global aggregation on the server.
  arXiv  Detail & Related papers  (2022-05-20T12:06:39Z)
• Unsupervised Domain Adaptive Salient Object Detection Through Uncertainty-Aware Pseudo-Label Learning [104.00026716576546]
  We propose to learn saliency from synthetic but clean labels, which naturally has higher pixel-labeling quality without the effort of manual annotations. We show that our proposed method outperforms the existing state-of-the-art deep unsupervised SOD methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2022-02-26T16:03:55Z)
• Feature Diversity Learning with Sample Dropout for Unsupervised Domain Adaptive Person Re-identification [0.0]
  This paper proposes a new approach to learn the feature representation with better generalization ability through limiting noisy pseudo labels. We put forward a brand-new method referred as to Feature Diversity Learning (FDL) under the classic mutual-teaching architecture. Experimental results show that our proposed FDL-SD achieves the state-of-the-art performance on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2022-01-25T10:10:48Z)
• Open-set Label Noise Can Improve Robustness Against Inherent Label Noise [27.885927200376386]
  We show that open-set noisy labels can be non-toxic and even benefit the robustness against inherent noisy labels. We propose a simple yet effective regularization by introducing Open-set samples with Dynamic Noisy Labels (ODNL) into training.
  arXiv  Detail & Related papers  (2021-06-21T07:15:50Z)
• In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning [53.1047775185362]
  Pseudo-labeling (PL) is a general SSL approach that does not have this constraint but performs relatively poorly in its original formulation. We argue that PL underperforms due to the erroneous high confidence predictions from poorly calibrated models. We propose an uncertainty-aware pseudo-label selection (UPS) framework which improves pseudo labeling accuracy by drastically reducing the amount of noise encountered in the training process.
  arXiv  Detail & Related papers  (2021-01-15T23:29:57Z)
• Environment Inference for Invariant Learning [9.63004099102596]
  We propose EIIL, a framework for domain-invariant learning that incorporates Environment Inference. We show that EIIL outperforms invariant learning methods on the CMNIST benchmark without using environment labels. We also establish connections between EIIL and algorithmic fairness, which enables EIIL to improve accuracy and calibration in a fair prediction problem.
  arXiv  Detail & Related papers  (2020-10-14T17:11:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.