DADA: Differentiable Automatic Data Augmentation

• URL: http://arxiv.org/abs/2003.03780v3
• Date: Thu, 30 Jul 2020 14:32:24 GMT
• Title: DADA: Differentiable Automatic Data Augmentation
• Authors: Yonggang Li and Guosheng Hu and Yongtao Wang and Timothy Hospedales and Neil M. Robertson and Yongxin Yang
• Abstract summary: We propose Differentiable Automatic Data Augmentation (DADA) which dramatically reduces the cost. We conduct extensive experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets. Results show our DADA is at least one order of magnitude faster than the state-of-the-art while achieving very comparable accuracy.
• Score: 58.560309490774976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data augmentation (DA) techniques aim to increase data variability, and thus train deep networks with better generalisation. The pioneering AutoAugment automated the search for optimal DA policies with reinforcement learning. However, AutoAugment is extremely computationally expensive, limiting its wide applicability. Followup works such as Population Based Augmentation (PBA) and Fast AutoAugment improved efficiency, but their optimization speed remains a bottleneck. In this paper, we propose Differentiable Automatic Data Augmentation (DADA) which dramatically reduces the cost. DADA relaxes the discrete DA policy selection to a differentiable optimization problem via Gumbel-Softmax. In addition, we introduce an unbiased gradient estimator, RELAX, leading to an efficient and effective one-pass optimization strategy to learn an efficient and accurate DA policy. We conduct extensive experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets. Furthermore, we demonstrate the value of Auto DA in pre-training for downstream detection problems. Results show our DADA is at least one order of magnitude faster than the state-of-the-art while achieving very comparable accuracy. The code is available at https://github.com/VDIGPKU/DADA.

Related papers

• AutoFT: Learning an Objective for Robust Fine-Tuning [60.641186718253735]
  Foundation models encode rich representations that can be adapted to downstream tasks by fine-tuning. Current approaches to robust fine-tuning use hand-crafted regularization techniques. We propose AutoFT, a data-driven approach for robust fine-tuning.
  arXiv  Detail & Related papers  (2024-01-18T18:58:49Z)
• AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentation [6.625868719336385]
  Adversarial training (AT) is an effective defense against adversarial examples. Data augmentation (DA) was shown to be effective in mitigating robust overfitting if appropriately designed and optimized for AT. This work proposes a new method to automatically learn online, instance-wise, DA policies to improve robust generalization for AT.
  arXiv  Detail & Related papers  (2023-06-12T15:54:52Z)
• Efficient Few-Shot Object Detection via Knowledge Inheritance [62.36414544915032]
  Few-shot object detection (FSOD) aims at learning a generic detector that can adapt to unseen tasks with scarce training samples. We present an efficient pretrain-transfer framework (PTF) baseline with no computational increment. We also propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights.
  arXiv  Detail & Related papers  (2022-03-23T06:24:31Z)
• Automatic tuning of hyper-parameters of reinforcement learning algorithms using Bayesian optimization with behavioral cloning [0.0]
  In reinforcement learning (RL), the information content of data gathered by the learning agent is dependent on the setting of many hyper- parameters. In this work, a novel approach for autonomous hyper- parameter setting using Bayesian optimization is proposed. Experiments reveal promising results compared to other manual tweaking and optimization-based approaches.
  arXiv  Detail & Related papers  (2021-12-15T13:10:44Z)
• DAAS: Differentiable Architecture and Augmentation Policy Search [107.53318939844422]
  This work considers the possible coupling between neural architectures and data augmentation and proposes an effective algorithm jointly searching for them. Our approach achieves 97.91% accuracy on CIFAR-10 and 76.6% Top-1 accuracy on ImageNet dataset, showing the outstanding performance of our search algorithm.
  arXiv  Detail & Related papers  (2021-09-30T17:15:17Z)
• VSAC: Efficient and Accurate Estimator for H and F [68.65610177368617]
  VSAC is a RANSAC-type robust estimator with a number of novelties. It is significantly faster than all its predecessors and runs on average in 1-2 ms, on a CPU. It is two orders of magnitude faster and yet as precise as MAGSAC++, the currently most accurate estimator of two-view geometry.
  arXiv  Detail & Related papers  (2021-06-18T17:04:57Z)
• Direct Differentiable Augmentation Search [25.177623230408656]
  We propose an efficient differentiable search algorithm called Direct Differentiable Augmentation Search (DDAS) It exploits meta-learning with one-step gradient update and continuous relaxation to the expected training loss for efficient search. Our DDAS achieves state-of-the-art performance and efficiency tradeoff while reducing the search cost dramatically.
  arXiv  Detail & Related papers  (2021-04-09T10:02:24Z)
• AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation [3.118384520557952]
  AutoAugment has sparked an interest in automated augmentation methods for deep learning models. We show that those methods are not robust when applied to biased and noisy data. We reformulate AutoAugment as a generalized automated dataset optimization (AutoDO) task. Our experiments show up to 9.3% improvement for biased datasets with label noise compared to prior methods.
  arXiv  Detail & Related papers  (2021-03-10T04:05:33Z)
• Improving Auto-Augment via Augmentation-Wise Weight Sharing [123.71986174280741]
  A key component of automatic augmentation search is the evaluation process for a particular augmentation policy. In this paper, we dive into the dynamics of augmented training of the model. We design a powerful and efficient proxy task based on the Augmentation-Wise Weight Sharing (AWS) to form a fast yet accurate evaluation process.
  arXiv  Detail & Related papers  (2020-09-30T15:23:12Z)
• Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving [35.44498286245894]
  We present an efficient sampling-based maximum-entropy inverse reinforcement learning (IRL) algorithm in this paper. We evaluate the proposed algorithm on real driving data, including both non-interactive and interactive scenarios.
  arXiv  Detail & Related papers  (2020-06-22T01:41:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.