Improving Auto-Augment via Augmentation-Wise Weight Sharing

• URL: http://arxiv.org/abs/2009.14737v2
• Date: Thu, 22 Oct 2020 15:12:47 GMT
• Title: Improving Auto-Augment via Augmentation-Wise Weight Sharing
• Authors: Keyu Tian, Chen Lin, Ming Sun, Luping Zhou, Junjie Yan, Wanli Ouyang
• Abstract summary: A key component of automatic augmentation search is the evaluation process for a particular augmentation policy. In this paper, we dive into the dynamics of augmented training of the model. We design a powerful and efficient proxy task based on the Augmentation-Wise Weight Sharing (AWS) to form a fast yet accurate evaluation process.
• Score: 123.71986174280741
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent progress on automatically searching augmentation policies has boosted the performance substantially for various tasks. A key component of automatic augmentation search is the evaluation process for a particular augmentation policy, which is utilized to return reward and usually runs thousands of times. A plain evaluation process, which includes full model training and validation, would be time-consuming. To achieve efficiency, many choose to sacrifice evaluation reliability for speed. In this paper, we dive into the dynamics of augmented training of the model. This inspires us to design a powerful and efficient proxy task based on the Augmentation-Wise Weight Sharing (AWS) to form a fast yet accurate evaluation process in an elegant way. Comprehensive analysis verifies the superiority of this approach in terms of effectiveness and efficiency. The augmentation policies found by our method achieve superior accuracies compared with existing auto-augmentation search methods. On CIFAR-10, we achieve a top-1 error rate of 1.24%, which is currently the best performing single model without extra training data. On ImageNet, we get a top-1 error rate of 20.36% for ResNet-50, which leads to 3.34% absolute error rate reduction over the baseline augmentation.

Related papers

• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning [55.96599486604344]
  We introduce an approach aimed at enhancing the reasoning capabilities of Large Language Models (LLMs) through an iterative preference learning process. We use Monte Carlo Tree Search (MCTS) to iteratively collect preference data, utilizing its look-ahead ability to break down instance-level rewards into more granular step-level signals. The proposed algorithm employs Direct Preference Optimization (DPO) to update the LLM policy using this newly generated step-level preference data.
  arXiv  Detail & Related papers  (2024-05-01T11:10:24Z)
• LA3: Efficient Label-Aware AutoAugment [23.705059658590436]
  We propose a novel two-stage data augmentation algorithm, named Label-Aware AutoAugment (LA3), which takes advantage of the label information. LA3 consists of two learning stages, where in the first stage, individual augmentation methods are evaluated and ranked for each label. In the second stage, a composite augmentation policy is constructed out of a selection of effective as well as complementary augmentations, which produces significant performance boost.
  arXiv  Detail & Related papers  (2023-04-20T13:42:18Z)
• Soft Augmentation for Image Classification [68.71067594724663]
  We propose generalizing augmentation with invariant transforms to soft augmentation. We show that soft targets allow for more aggressive data augmentation. We also show that soft augmentations generalize to self-supervised classification tasks.
  arXiv  Detail & Related papers  (2022-11-09T01:04:06Z)
• Efficient Few-Shot Object Detection via Knowledge Inheritance [62.36414544915032]
  Few-shot object detection (FSOD) aims at learning a generic detector that can adapt to unseen tasks with scarce training samples. We present an efficient pretrain-transfer framework (PTF) baseline with no computational increment. We also propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights.
  arXiv  Detail & Related papers  (2022-03-23T06:24:31Z)
• Reinforcement Learning in the Wild: Scalable RL Dispatching Algorithm Deployed in Ridehailing Marketplace [12.298997392937876]
  This study proposes a real-time dispatching algorithm based on reinforcement learning. It is deployed online in multiple cities under DiDi's operation for A/B testing and is launched in one of the major international markets. The deployed algorithm shows over 1.3% improvement in total driver income from A/B testing.
  arXiv  Detail & Related papers  (2022-02-10T16:07:17Z)
• Bag of Tricks for Adversarial Training [50.53525358778331]
  Adrial training is one of the most effective strategies for promoting model robustness. Recent benchmarks show that most of the proposed improvements on AT are less effective than simply early stopping the training procedure.
  arXiv  Detail & Related papers  (2020-10-01T15:03:51Z)
• Hypernetwork-Based Augmentation [1.6752182911522517]
  We propose an efficient gradient-based search algorithm, called Hypernetwork-Based Augmentation (HBA) Our HBA uses a hypernetwork to approximate a population-based training algorithm. Our results show that HBA is competitive to the state-of-the-art methods in terms of both search speed and accuracy.
  arXiv  Detail & Related papers  (2020-06-11T10:36:39Z)
• DADA: Differentiable Automatic Data Augmentation [58.560309490774976]
  We propose Differentiable Automatic Data Augmentation (DADA) which dramatically reduces the cost. We conduct extensive experiments on CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets. Results show our DADA is at least one order of magnitude faster than the state-of-the-art while achieving very comparable accuracy.
  arXiv  Detail & Related papers  (2020-03-08T13:23:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.