WeMix: How to Better Utilize Data Augmentation
- URL: http://arxiv.org/abs/2010.01267v1
- Date: Sat, 3 Oct 2020 03:12:18 GMT
- Title: WeMix: How to Better Utilize Data Augmentation
- Authors: Yi Xu, Asaf Noy, Ming Lin, Qi Qian, Hao Li, Rong Jin
- Abstract summary: We develop a comprehensive analysis that reveals pros and cons of data augmentation.
The main limitation of data augmentation arises from the data bias.
We develop two novel algorithms, termed "AugDrop" and "MixLoss", to correct the data bias in the data augmentation.
- Score: 36.07712244423405
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Data augmentation is a widely used training trick in deep learning to improve
the network generalization ability. Despite many encouraging results, several
recent studies did point out limitations of the conventional data augmentation
scheme in certain scenarios, calling for a better theoretical understanding of
data augmentation. In this work, we develop a comprehensive analysis that
reveals pros and cons of data augmentation. The main limitation of data
augmentation arises from the data bias, i.e. the augmented data distribution
can be quite different from the original one. This data bias leads to a
suboptimal performance of existing data augmentation methods. To this end, we
develop two novel algorithms, termed "AugDrop" and "MixLoss", to correct the
data bias in the data augmentation. Our theoretical analysis shows that both
algorithms are guaranteed to improve the effect of data augmentation through
the bias correction, which is further validated by our empirical studies.
Finally, we propose a generic algorithm "WeMix" by combining AugDrop and
MixLoss, whose effectiveness is observed from extensive empirical evaluations.
Related papers
- Understanding when Dynamics-Invariant Data Augmentations Benefit Model-Free Reinforcement Learning Updates [3.5253513747455303]
We identify general aspects of data augmentation (DA) responsible for observed learning improvements.
Our study focuses on sparse-reward tasks with dynamics-invariant data augmentation functions.
arXiv Detail & Related papers (2023-10-26T21:28:50Z) - DualAug: Exploiting Additional Heavy Augmentation with OOD Data
Rejection [77.6648187359111]
We propose a novel data augmentation method, named textbfDualAug, to keep the augmentation in distribution as much as possible at a reasonable time and computational cost.
Experiments on supervised image classification benchmarks show that DualAug improve various automated data augmentation method.
arXiv Detail & Related papers (2023-10-12T08:55:10Z) - On Counterfactual Data Augmentation Under Confounding [30.76982059341284]
Counterfactual data augmentation has emerged as a method to mitigate confounding biases in the training data.
These biases arise due to various observed and unobserved confounding variables in the data generation process.
We show how our simple augmentation method helps existing state-of-the-art methods achieve good results.
arXiv Detail & Related papers (2023-05-29T16:20:23Z) - Is augmentation effective to improve prediction in imbalanced text
datasets? [3.1690891866882236]
We argue that adjusting the cutoffs without data augmentation can produce similar results to oversampling techniques.
Our findings contribute to a better understanding of the strengths and limitations of different approaches to dealing with imbalanced data.
arXiv Detail & Related papers (2023-04-20T13:07:31Z) - Advanced Data Augmentation Approaches: A Comprehensive Survey and Future
directions [57.30984060215482]
We provide a background of data augmentation, a novel and comprehensive taxonomy of reviewed data augmentation techniques, and the strengths and weaknesses (wherever possible) of each technique.
We also provide comprehensive results of the data augmentation effect on three popular computer vision tasks, such as image classification, object detection and semantic segmentation.
arXiv Detail & Related papers (2023-01-07T11:37:32Z) - Automatic Data Augmentation via Invariance-Constrained Learning [94.27081585149836]
Underlying data structures are often exploited to improve the solution of learning tasks.
Data augmentation induces these symmetries during training by applying multiple transformations to the input data.
This work tackles these issues by automatically adapting the data augmentation while solving the learning task.
arXiv Detail & Related papers (2022-09-29T18:11:01Z) - MixAugment & Mixup: Augmentation Methods for Facial Expression
Recognition [4.273075747204267]
We propose a new data augmentation strategy which is based on Mixup, called MixAugment.
We conduct an extensive experimental study that proves the effectiveness of MixAugment over Mixup and various state-of-the-art methods.
arXiv Detail & Related papers (2022-05-09T17:43:08Z) - Invariance Learning in Deep Neural Networks with Differentiable Laplace
Approximations [76.82124752950148]
We develop a convenient gradient-based method for selecting the data augmentation.
We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
arXiv Detail & Related papers (2022-02-22T02:51:11Z) - On the Benefits of Invariance in Neural Networks [56.362579457990094]
We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation.
We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
arXiv Detail & Related papers (2020-05-01T02:08:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.