Speeding Up Image Classifiers with Little Companions

• URL: http://arxiv.org/abs/2406.17117v2
• Date: Thu, 27 Jun 2024 03:25:19 GMT
• Title: Speeding Up Image Classifiers with Little Companions
• Authors: Yang Liu, Kowshik Thopalli, Jayaraman Thiagarajan,
• Abstract summary: Scaling up neural networks has been a key recipe to the success of large language and vision models. We develop a simple model-a two-pass Little-Big that first uses a light-weight "little" model to make predictions of all samples. Little-Big also speeds up the InternImage-G-512 while achieving 90% ImageNet-1K top-1 accuracy.
• Score: 5.9999780224657195
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Scaling up neural networks has been a key recipe to the success of large language and vision models. However, in practice, up-scaled models can be disproportionately costly in terms of computations, providing only marginal improvements in performance; for example, EfficientViT-L3-384 achieves <2% improvement on ImageNet-1K accuracy over the base L1-224 model, while requiring $14\times$ more multiply-accumulate operations (MACs). In this paper, we investigate scaling properties of popular families of neural networks for image classification, and find that scaled-up models mostly help with "difficult" samples. Decomposing the samples by difficulty, we develop a simple model-agnostic two-pass Little-Big algorithm that first uses a light-weight "little" model to make predictions of all samples, and only passes the difficult ones for the "big" model to solve. Good little companion achieve drastic MACs reduction for a wide variety of model families and scales. Without loss of accuracy or modification of existing models, our Little-Big models achieve MACs reductions of 76% for EfficientViT-L3-384, 81% for EfficientNet-B7-600, 71% for DeiT3-L-384 on ImageNet-1K. Little-Big also speeds up the InternImage-G-512 model by 62% while achieving 90% ImageNet-1K top-1 accuracy, serving both as a strong baseline and as a simple practical method for large model compression.

Related papers

• Tiny Models are the Computational Saver for Large Models [1.8350044465969415]
  This paper introduces TinySaver, an early-exit-like dynamic model compression approach which employs tiny models to substitute large models adaptively. Our evaluation of this approach in ImageNet-1k classification demonstrates its potential to reduce the number of compute operations by up to 90%, with only negligible losses in performance.
  arXiv  Detail & Related papers  (2024-03-26T14:14:30Z)
• A Simple and Efficient Baseline for Data Attribution on Images [107.12337511216228]
  Current state-of-the-art approaches require a large ensemble of as many as 300,000 models to accurately attribute model predictions. In this work, we focus on a minimalist baseline, utilizing the feature space of a backbone pretrained via self-supervised learning to perform data attribution. Our method is model-agnostic and scales easily to large datasets.
  arXiv  Detail & Related papers  (2023-11-03T17:29:46Z)
• Core Risk Minimization using Salient ImageNet [53.616101711801484]
  We introduce the Salient Imagenet dataset with more than 1 million soft masks localizing core and spurious features for all 1000 Imagenet classes. Using this dataset, we first evaluate the reliance of several Imagenet pretrained models (42 total) on spurious features. Next, we introduce a new learning paradigm called Core Risk Minimization (CoRM) whose objective ensures that the model predicts a class using its core features.
  arXiv  Detail & Related papers  (2022-03-28T01:53:34Z)
• Combined Scaling for Zero-shot Transfer Learning [146.0851484769142]
  We present a combined scaling method - named BASIC - that achieves 85.7% top-1 accuracy on the ImageNet ILSVRC-2012 validation set. This accuracy surpasses best published similar models - CLIP and ALIGN - by 9.3%. Our model also shows significant improvements in robustness benchmarks.
  arXiv  Detail & Related papers  (2021-11-19T05:25:46Z)
• SimMIM: A Simple Framework for Masked Image Modeling [29.015777125540613]
  This paper presents Sim, a simple framework for masked image modeling. We study the major components in our framework, and find that simple designs of each component have revealed very strong representation learning performance. We also leverage this approach to facilitate the training of a 3B model, that by $40times$ less data than that in previous practice, we achieve the state-of-the-art on four representative vision benchmarks.
  arXiv  Detail & Related papers  (2021-11-18T18:59:45Z)
• Network Augmentation for Tiny Deep Learning [73.57192520534585]
  We introduce Network Augmentation (NetAug), a new training method for improving the performance of tiny neural networks. We demonstrate the effectiveness of NetAug on image classification and object detection.
  arXiv  Detail & Related papers  (2021-10-17T18:48:41Z)
• Greedy Network Enlarging [53.319011626986004]
  We propose a greedy network enlarging method based on the reallocation of computations. With step-by-step modifying the computations on different stages, the enlarged network will be equipped with optimal allocation and utilization of MACs. With application of our method on GhostNet, we achieve state-of-the-art 80.9% and 84.3% ImageNet top-1 accuracies.
  arXiv  Detail & Related papers  (2021-07-31T08:36:30Z)
• Effective Model Sparsification by Scheduled Grow-and-Prune Methods [73.03533268740605]
  We propose a novel scheduled grow-and-prune (GaP) methodology without pre-training the dense models. Experiments have shown that such models can match or beat the quality of highly optimized dense models at 80% sparsity on a variety of tasks.
  arXiv  Detail & Related papers  (2021-06-18T01:03:13Z)
• High-Performance Large-Scale Image Recognition Without Normalization [34.58818094675353]
  Batch normalization is a key component of most image classification models, but it has many undesirable properties. We develop an adaptive gradient clipping technique which overcomes these instabilities, and design a significantly improved class of Normalizer-Free ResNets. Our models attain significantly better performance than their batch-normalized counterparts when finetuning on ImageNet after large-scale pre-training.
  arXiv  Detail & Related papers  (2021-02-11T18:23:20Z)
• Scalable and Practical Natural Gradient for Large-Scale Deep Learning [19.220930193896404]
  SP-NGD scales to large mini-batch sizes with a negligible computational overhead as compared to first-order methods. We demonstrate convergence to a top-1 validation accuracy of 75.4% in 5.5 minutes using a mini-batch size of 32,768 with 1,024 GPUs, as well as an accuracy of 74.9% with an extremely large mini-batch size of 131,072 in 873 steps of SP-NGD.
  arXiv  Detail & Related papers  (2020-02-13T11:55:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.