Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data

• URL: http://arxiv.org/abs/2010.11966v1
• Date: Thu, 22 Oct 2020 18:01:51 GMT
• Title: Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data
• Authors: David Lowell, Brian E. Howard, Zachary C. Lipton, Byron C. Wallace
• Abstract summary: Unsupervised Data Augmentation (UDA) is a semi-supervised technique that applies a consistency loss to penalize differences between a model's predictions. In this paper, we re-examine UDA and demonstrate its efficacy on several sequential tasks. We find that applying its consistency loss affords meaningful gains without any unlabeled data at all.
• Score: 40.82826366059613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised Data Augmentation (UDA) is a semi-supervised technique that applies a consistency loss to penalize differences between a model's predictions on (a) observed (unlabeled) examples; and (b) corresponding 'noised' examples produced via data augmentation. While UDA has gained popularity for text classification, open questions linger over which design decisions are necessary and over how to extend the method to sequence labeling tasks. This method has recently gained traction for text classification. In this paper, we re-examine UDA and demonstrate its efficacy on several sequential tasks. Our main contribution is an empirical study of UDA to establish which components of the algorithm confer benefits in NLP. Notably, although prior work has emphasized the use of clever augmentation techniques including back-translation, we find that enforcing consistency between predictions assigned to observed and randomly substituted words often yields comparable (or greater) benefits compared to these complex perturbation models. Furthermore, we find that applying its consistency loss affords meaningful gains without any unlabeled data at all, i.e., in a standard supervised setting. In short: UDA need not be unsupervised, and does not require complex data augmentation to be effective.

Related papers

• Unsupervised Transfer Learning via Adversarial Contrastive Training [3.227277661633986]
  We propose a novel unsupervised transfer learning approach using adversarial contrastive training (ACT) Our experimental results demonstrate outstanding classification accuracy with both fine-tuned linear probe and K-NN protocol across various datasets.
  arXiv  Detail & Related papers  (2024-08-16T05:11:52Z)
• Conditional Semi-Supervised Data Augmentation for Spam Message Detection with Low Resource Data [0.0]
  We propose a conditional semi-supervised data augmentation for a spam detection model lacking the availability of data. We exploit unlabeled data for data augmentation to extend training data. Latent variables can come from labeled and unlabeled data as the input for the final classifier.
  arXiv  Detail & Related papers  (2024-07-06T07:51:24Z)
• Bias Challenges in Counterfactual Data Augmentation [17.568839986755744]
  Deep learning models tend not to be out-of-distribution robust due to their reliance on spurious features to solve the task. Counterfactual data augmentations provide a general way of achieving representations that are counterfactual-invariant to spurious features. We show that counterfactual data augmentations may not achieve the desired counterfactual-invariance if the augmentation is performed by a context-guessing machine.
  arXiv  Detail & Related papers  (2022-09-12T09:17:49Z)
• Augmentation-Aware Self-Supervision for Data-Efficient GAN Training [68.81471633374393]
  Training generative adversarial networks (GANs) with limited data is challenging because the discriminator is prone to overfitting. We propose a novel augmentation-aware self-supervised discriminator that predicts the augmentation parameter of the augmented data. We compare our method with state-of-the-art (SOTA) methods using the class-conditional BigGAN and unconditional StyleGAN2 architectures.
  arXiv  Detail & Related papers  (2022-05-31T10:35:55Z)
• Self-Trained One-class Classification for Unsupervised Anomaly Detection [56.35424872736276]
  Anomaly detection (AD) has various applications across domains, from manufacturing to healthcare. In this work, we focus on unsupervised AD problems whose entire training data are unlabeled and may contain both normal and anomalous samples. To tackle this problem, we build a robust one-class classification framework via data refinement. We show that our method outperforms state-of-the-art one-class classification method by 6.3 AUC and 12.5 average precision.
  arXiv  Detail & Related papers  (2021-06-11T01:36:08Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)
• Semi-Supervised Models via Data Augmentationfor Classifying Interactive Affective Responses [85.04362095899656]
  We present semi-supervised models with data augmentation (SMDA), a semi-supervised text classification system to classify interactive affective responses. For labeled sentences, we performed data augmentations to uniform the label distributions and computed supervised loss during training process. For unlabeled sentences, we explored self-training by regarding low-entropy predictions over unlabeled sentences as pseudo labels.
  arXiv  Detail & Related papers  (2020-04-23T05:02:31Z)
• MixPUL: Consistency-based Augmentation for Positive and Unlabeled Learning [8.7382177147041]
  We propose a simple yet effective data augmentation method, coinedalgo, based on emphconsistency regularization. algoincorporates supervised and unsupervised consistency training to generate augmented data. We show thatalgoachieves an averaged improvement of classification error from 16.49 to 13.09 on the CIFAR-10 dataset across different positive data amount.
  arXiv  Detail & Related papers  (2020-04-20T15:43:33Z)
• Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data [87.61504710345528]
  We propose two strategies for freeing a neural network from tuning with OoD data, while improving its OoD detection performance. We specifically propose to decompose confidence scoring as well as a modified input pre-processing method. Our further analysis on a larger scale image dataset shows that the two types of distribution shifts, specifically semantic shift and non-semantic shift, present a significant difference.
  arXiv  Detail & Related papers  (2020-02-26T04:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.