Revisiting Random Channel Pruning for Neural Network Compression

• URL: http://arxiv.org/abs/2205.05676v1
• Date: Wed, 11 May 2022 17:59:04 GMT
• Title: Revisiting Random Channel Pruning for Neural Network Compression
• Authors: Yawei Li, Kamil Adamczewski, Wen Li, Shuhang Gu, Radu Timofte, Luc Van Gool
• Abstract summary: Channel (or 3D filter) pruning serves as an effective way to accelerate the inference of neural networks. In this paper, we try to determine the channel configuration of the pruned models by random search. We show that this simple strategy works quite well compared with other channel pruning methods.
• Score: 159.99002793644163
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Channel (or 3D filter) pruning serves as an effective way to accelerate the inference of neural networks. There has been a flurry of algorithms that try to solve this practical problem, each being claimed effective in some ways. Yet, a benchmark to compare those algorithms directly is lacking, mainly due to the complexity of the algorithms and some custom settings such as the particular network configuration or training procedure. A fair benchmark is important for the further development of channel pruning. Meanwhile, recent investigations reveal that the channel configurations discovered by pruning algorithms are at least as important as the pre-trained weights. This gives channel pruning a new role, namely searching the optimal channel configuration. In this paper, we try to determine the channel configuration of the pruned models by random search. The proposed approach provides a new way to compare different methods, namely how well they behave compared with random pruning. We show that this simple strategy works quite well compared with other channel pruning methods. We also show that under this setting, there are surprisingly no clear winners among different channel importance evaluation methods, which then may tilt the research efforts into advanced channel configuration searching methods.

Related papers

• Network Pruning via Feature Shift Minimization [8.593369249204132]
  We propose a novel Feature Shift Minimization (FSM) method to compress CNN models, which evaluates the feature shift by converging the information of both features and filters. The proposed method yields state-of-the-art performance on various benchmark networks and datasets, verified by extensive experiments.
  arXiv  Detail & Related papers  (2022-07-06T12:50:26Z)
• AdaPruner: Adaptive Channel Pruning and Effective Weights Inheritance [9.3421559369389]
  We propose a pruning framework that adaptively determines the number of each layer's channels as well as the wights inheritance criteria for sub-network. AdaPruner allows to obtain pruned network quickly, accurately and efficiently. On ImageNet, we reduce 32.8% FLOPs of MobileNetV2 with only 0.62% decrease for top-1 accuracy, which exceeds all previous state-of-the-art channel pruning methods.
  arXiv  Detail & Related papers  (2021-09-14T01:52:05Z)
• Group Fisher Pruning for Practical Network Compression [58.25776612812883]
  We present a general channel pruning approach that can be applied to various complicated structures. We derive a unified metric based on Fisher information to evaluate the importance of a single channel and coupled channels. Our method can be used to prune any structures including those with coupled channels.
  arXiv  Detail & Related papers  (2021-08-02T08:21:44Z)
• Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot [55.37967301483917]
  Conventional wisdom of pruning algorithms suggests that pruning methods exploit information from training data to find goodworks. In this paper, we conduct sanity checks for the above beliefs on several recent unstructured pruning methods. We propose a series of simple emphdata-independent prune ratios for each layer, and randomly prune each layer accordingly to get a subnetwork.
  arXiv  Detail & Related papers  (2020-09-22T17:36:17Z)
• Operation-Aware Soft Channel Pruning using Differentiable Masks [51.04085547997066]
  We propose a data-driven algorithm, which compresses deep neural networks in a differentiable way by exploiting the characteristics of operations. We perform extensive experiments and achieve outstanding performance in terms of the accuracy of output networks.
  arXiv  Detail & Related papers  (2020-07-08T07:44:00Z)
• DMCP: Differentiable Markov Channel Pruning for Neural Networks [67.51334229530273]
  We propose a novel differentiable method for channel pruning, named Differentiable Markov Channel Pruning (DMCP) Our method is differentiable and can be directly optimized by gradient descent with respect to standard task loss and budget regularization. To validate the effectiveness of our method, we perform extensive experiments on Imagenet with ResNet and MobilenetV2.
  arXiv  Detail & Related papers  (2020-05-07T09:39:55Z)
• Channel Pruning via Automatic Structure Search [109.83642249625098]
  We propose a new channel pruning method based on artificial bee colony algorithm (ABC), dubbed as ABCPruner. ABCPruner has been demonstrated to be more effective, which also enables the fine-tuning to be conducted efficiently in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-01-23T14:51:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.