Joint Search of Data Augmentation Policies and Network Architectures

• URL: http://arxiv.org/abs/2012.09407v2
• Date: Tue, 12 Jan 2021 11:59:50 GMT
• Title: Joint Search of Data Augmentation Policies and Network Architectures
• Authors: Taiga Kashima, Yoshihiro Yamada, Shunta Saito
• Abstract summary: The proposed method combines differentiable methods for augmentation policy search and network architecture search to jointly optimize them in the end-to-end manner. experimental results show our method achieves competitive or superior performance to the independently searched results.
• Score: 4.887917220146243
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The common pipeline of training deep neural networks consists of several building blocks such as data augmentation and network architecture selection. AutoML is a research field that aims at automatically designing those parts, but most methods explore each part independently because it is more challenging to simultaneously search all the parts. In this paper, we propose a joint optimization method for data augmentation policies and network architectures to bring more automation to the design of training pipeline. The core idea of our approach is to make the whole part differentiable. The proposed method combines differentiable methods for augmentation policy search and network architecture search to jointly optimize them in the end-to-end manner. The experimental results show our method achieves competitive or superior performance to the independently searched results.

Related papers

• EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition [54.99121380536659]
  Eye movement biometrics have received increasing attention thanks to its high secure identification. Deep learning (DL) models have been recently successfully applied for eye movement recognition. DL architecture still is determined by human prior knowledge. We propose EM-DARTS, a hierarchical differentiable architecture search algorithm to automatically design the DL architecture for eye movement recognition.
  arXiv  Detail & Related papers  (2024-09-22T13:11:08Z)
• Pareto-aware Neural Architecture Generation for Diverse Computational Budgets [94.27982238384847]
  Existing methods often perform an independent architecture search process for each target budget. We propose a Neural Architecture Generator (PNAG) which only needs to be trained once and dynamically produces the optimal architecture for any given budget via inference. Such a joint search algorithm not only greatly reduces the overall search cost but also improves the results.
  arXiv  Detail & Related papers  (2022-10-14T08:30:59Z)
• Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR [1.2507285499419876]
  We present an automatic partitioner that identifies efficient combinations for many model architectures and accelerator systems. Our key findings are that a Monte Carlo Tree Search-based partitioner leveraging partition-specific compiler analysis directly into the search and guided goals matches expert-level strategies for various models.
  arXiv  Detail & Related papers  (2022-10-07T17:46:46Z)
• Efficient Search of Multiple Neural Architectures with Different Complexities via Importance Sampling [3.759936323189417]
  This study focuses on the architecture complexity-aware one-shot NAS that optimize the objective function composed of the weighted sum of two metrics. The proposed method is applied to the architecture search of convolutional neural networks on the CIAFR-10 and ImageNet datasets.
  arXiv  Detail & Related papers  (2022-07-21T07:06:03Z)
• Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization [50.50023451369742]
  Pruning-as-Search (PaS) is an end-to-end channel pruning method to search out desired sub-network automatically and efficiently. Our proposed architecture outperforms prior arts by around $1.0%$ top-1 accuracy on ImageNet-1000 classification task.
  arXiv  Detail & Related papers  (2022-06-02T17:58:54Z)
• DAAS: Differentiable Architecture and Augmentation Policy Search [107.53318939844422]
  This work considers the possible coupling between neural architectures and data augmentation and proposes an effective algorithm jointly searching for them. Our approach achieves 97.91% accuracy on CIFAR-10 and 76.6% Top-1 accuracy on ImageNet dataset, showing the outstanding performance of our search algorithm.
  arXiv  Detail & Related papers  (2021-09-30T17:15:17Z)
• CHASE: Robust Visual Tracking via Cell-Level Differentiable Neural Architecture Search [14.702573109803307]
  We propose a novel cell-level differentiable architecture search mechanism to automate the network design of the tracking module. The proposed approach is simple, efficient, and with no need to stack a series of modules to construct a network. Our approach is easy to be incorporated into existing trackers, which is empirically validated using different differentiable architecture search-based methods and tracking objectives.
  arXiv  Detail & Related papers  (2021-07-02T15:16:45Z)
• Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components [71.03032589756434]
  We investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. We characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.
  arXiv  Detail & Related papers  (2021-06-16T17:12:26Z)
• Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search [60.965024145243596]
  One-shot weight sharing methods have recently drawn great attention in neural architecture search due to high efficiency and competitive performance. To alleviate this problem, we present a simple yet effective architecture distillation method. We introduce the concept of prioritized path, which refers to the architecture candidates exhibiting superior performance during training. Since the prioritized paths are changed on the fly depending on their performance and complexity, the final obtained paths are the cream of the crop.
  arXiv  Detail & Related papers  (2020-10-29T17:55:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.