Fixing the train-test resolution discrepancy: FixEfficientNet

• URL: http://arxiv.org/abs/2003.08237v5
• Date: Wed, 18 Nov 2020 09:56:31 GMT
• Title: Fixing the train-test resolution discrepancy: FixEfficientNet
• Authors: Hugo Touvron, Andrea Vedaldi, Matthijs Douze, Herv\'e J\'egou
• Abstract summary: This paper provides an analysis of the performance of the EfficientNet image classifiers with several recent training procedures. The resulting network, called FixEfficientNet, significantly outperforms the initial architecture with the same number of parameters.
• Score: 98.64315617109344
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper provides an extensive analysis of the performance of the EfficientNet image classifiers with several recent training procedures, in particular one that corrects the discrepancy between train and test images. The resulting network, called FixEfficientNet, significantly outperforms the initial architecture with the same number of parameters. For instance, our FixEfficientNet-B0 trained without additional training data achieves 79.3% top-1 accuracy on ImageNet with 5.3M parameters. This is a +0.5% absolute improvement over the Noisy student EfficientNet-B0 trained with 300M unlabeled images. An EfficientNet-L2 pre-trained with weak supervision on 300M unlabeled images and further optimized with FixRes achieves 88.5% top-1 accuracy (top-5: 98.7%), which establishes the new state of the art for ImageNet with a single crop. These improvements are thoroughly evaluated with cleaner protocols than the one usually employed for Imagenet, and particular we show that our improvement remains in the experimental setting of ImageNet-v2, that is less prone to overfitting, and with ImageNet Real Labels. In both cases we also establish the new state of the art.

Related papers

• Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement [68.44100784364987]
  We propose a strategy to improve a dataset once such that the accuracy of any model architecture trained on the reinforced dataset is improved at no additional training cost for users. We create a reinforced version of the ImageNet training dataset, called ImageNet+, as well as reinforced datasets CIFAR-100+, Flowers-102+, and Food-101+. Models trained with ImageNet+ are more accurate, robust, and calibrated, and transfer well to downstream tasks.
  arXiv  Detail & Related papers  (2023-03-15T23:10:17Z)
• To be Critical: Self-Calibrated Weakly Supervised Learning for Salient Object Detection [95.21700830273221]
  Weakly-supervised salient object detection (WSOD) aims to develop saliency models using image-level annotations. We propose a self-calibrated training strategy by explicitly establishing a mutual calibration loop between pseudo labels and network predictions. We prove that even a much smaller dataset with well-matched annotations can facilitate models to achieve better performance as well as generalizability.
  arXiv  Detail & Related papers  (2021-09-04T02:45:22Z)
• Post-training deep neural network pruning via layer-wise calibration [70.65691136625514]
  We propose a data-free extension of the approach for computer vision models based on automatically-generated synthetic fractal images. When using real data, we are able to get a ResNet50 model on ImageNet with 65% sparsity rate in 8-bit precision in a post-training setting.
  arXiv  Detail & Related papers  (2021-04-30T14:20:51Z)
• EfficientNetV2: Smaller Models and Faster Training [91.77432224225221]
  This paper introduces EfficientNetV2, a new family of convolutional networks that have faster training speed and better parameter efficiency than previous models. We use a combination of training-aware neural architecture search and scaling, to jointly optimize training speed and parameter efficiency. Our experiments show that EfficientNetV2 models train much faster than state-of-the-art models while being up to 6.8x smaller.
  arXiv  Detail & Related papers  (2021-04-01T07:08:36Z)
• High-Performance Large-Scale Image Recognition Without Normalization [34.58818094675353]
  Batch normalization is a key component of most image classification models, but it has many undesirable properties. We develop an adaptive gradient clipping technique which overcomes these instabilities, and design a significantly improved class of Normalizer-Free ResNets. Our models attain significantly better performance than their batch-normalized counterparts when finetuning on ImageNet after large-scale pre-training.
  arXiv  Detail & Related papers  (2021-02-11T18:23:20Z)
• Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour [38.89981855438478]
  We present techniques to scale up the training of EfficientNets on TPU-v3 Pods with 2048 cores. We are able to train EfficientNet on ImageNet to an accuracy of 83% in 1 hour and 4 minutes.
  arXiv  Detail & Related papers  (2020-10-30T19:27:11Z)
• MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks [57.69809561405253]
  We introduce a framework that is able to boost the vanilla ResNet-50 to 80%+ Top-1 accuracy on ImageNet without tricks. Our method obtains 80.67% top-1 accuracy on ImageNet using a single crop-size of 224x224 with vanilla ResNet-50. Our framework consistently improves from 69.76% to 73.19% on smaller ResNet-18.
  arXiv  Detail & Related papers  (2020-09-17T17:59:33Z)
• Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network [6.938261599173859]
  We show how to improve the accuracy and robustness of basic CNN models. Our proposed assembled ResNet-50 shows improvements in top-1 accuracy from 76.3% to 82.78%, mCE from 76.0% to 48.9% and mFR from 57.7% to 32.3%. Our approach achieved 1st place in the iFood Competition Fine-Grained Visual Recognition at CVPR 2019.
  arXiv  Detail & Related papers  (2020-01-17T12:42:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.