Sub-network Multi-objective Evolutionary Algorithm for Filter Pruning
- URL: http://arxiv.org/abs/2211.01957v1
- Date: Sat, 22 Oct 2022 13:34:14 GMT
- Title: Sub-network Multi-objective Evolutionary Algorithm for Filter Pruning
- Authors: Xuhua Li, Weize Sun, Lei Huang, Shaowu Chen
- Abstract summary: Filter pruning is a common method to achieve model compression and acceleration in deep neural networks (DNNs)
We propose a Sub-network Multiobjective Evolutionary Algorithm (SMOEA) for filter pruning.
Experiments on VGG-14 model for CIFAR-10 verify the effectiveness of the proposed SMOEA.
- Score: 5.998027804346945
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Filter pruning is a common method to achieve model compression and
acceleration in deep neural networks (DNNs).Some research regarded filter
pruning as a combinatorial optimization problem and thus used evolutionary
algorithms (EA) to prune filters of DNNs. However, it is difficult to find a
satisfactory compromise solution in a reasonable time due to the complexity of
solution space searching. To solve this problem, we first formulate a
multi-objective optimization problem based on a sub-network of the full model
and propose a Sub-network Multiobjective Evolutionary Algorithm (SMOEA) for
filter pruning. By progressively pruning the convolutional layers in groups,
SMOEA can obtain a lightweight pruned result with better
performance.Experiments on VGG-14 model for CIFAR-10 verify the effectiveness
of the proposed SMOEA. Specifically, the accuracy of the pruned model with
16.56% parameters decreases by 0.28% only, which is better than the widely used
popular filter pruning criteria.
Related papers
- A Greedy Hierarchical Approach to Whole-Network Filter-Pruning in CNNs [2.188091591747149]
Whole-network filter pruning algorithms prune varying fractions of filters from each layer, hence providing greater flexibility.
This paper proposes a two-level hierarchical approach for whole-network filter pruning which is efficient and uses the classification loss as the final criterion.
Our method reduces the RAM requirement for ResNext101 from 7.6 GB to 1.5 GB and achieves a 94% reduction in FLOPS without losing accuracy on CIFAR-10.
arXiv Detail & Related papers (2024-08-22T03:59:57Z) - Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep
Neural Networks [48.089501687522954]
We propose a novel layer-adaptive weight-pruning approach for Deep Neural Networks (DNNs)
Our approach takes into account the collective influence of all layers to design a layer-adaptive pruning scheme.
Our experiments demonstrate the superiority of our approach over existing methods on the ImageNet and CIFAR-10 datasets.
arXiv Detail & Related papers (2023-08-21T03:22:47Z) - Filter Pruning for Efficient CNNs via Knowledge-driven Differential
Filter Sampler [103.97487121678276]
Filter pruning simultaneously accelerates the computation and reduces the memory overhead of CNNs.
We propose a novel Knowledge-driven Differential Filter Sampler(KDFS) with Masked Filter Modeling(MFM) framework for filter pruning.
arXiv Detail & Related papers (2023-07-01T02:28:41Z) - Asymptotic Soft Cluster Pruning for Deep Neural Networks [5.311178623385279]
Filter pruning method introduces structural sparsity by removing selected filters.
We propose a novel filter pruning method called Asymptotic Soft Cluster Pruning.
Our method can achieve competitive results compared with many state-of-the-art algorithms.
arXiv Detail & Related papers (2022-06-16T13:58:58Z) - A Passive Similarity based CNN Filter Pruning for Efficient Acoustic
Scene Classification [23.661189257759535]
We present a method to develop low-complexity convolutional neural networks (CNNs) for acoustic scene classification (ASC)
We propose a passive filter pruning framework, where a few convolutional filters from the CNNs are eliminated to yield compressed CNNs.
The proposed method is simple, reduces computations per inference by 27%, with 25% fewer parameters, with less than 1% drop in accuracy.
arXiv Detail & Related papers (2022-03-29T17:00:06Z) - Improve Convolutional Neural Network Pruning by Maximizing Filter
Variety [0.0]
Neural network pruning is a widely used strategy for reducing model storage and computing requirements.
Common pruning criteria, such as l1-norm or movement, usually do not consider the individual utility of filters.
We present a technique solving those two issues, and which can be appended to any pruning criteria.
arXiv Detail & Related papers (2022-03-11T09:00:59Z) - Filter-enhanced MLP is All You Need for Sequential Recommendation [89.0974365344997]
In online platforms, logged user behavior data is inevitable to contain noise.
We borrow the idea of filtering algorithms from signal processing that attenuates the noise in the frequency domain.
We propose textbfFMLP-Rec, an all-MLP model with learnable filters for sequential recommendation task.
arXiv Detail & Related papers (2022-02-28T05:49:35Z) - Batch Normalization Tells You Which Filter is Important [49.903610684578716]
We propose a simple yet effective filter pruning method by evaluating the importance of each filter based on the BN parameters of pre-trained CNNs.
The experimental results on CIFAR-10 and ImageNet demonstrate that the proposed method can achieve outstanding performance.
arXiv Detail & Related papers (2021-12-02T12:04:59Z) - Dependency Aware Filter Pruning [74.69495455411987]
Pruning a proportion of unimportant filters is an efficient way to mitigate the inference cost.
Previous work prunes filters according to their weight norms or the corresponding batch-norm scaling factors.
We propose a novel mechanism to dynamically control the sparsity-inducing regularization so as to achieve the desired sparsity.
arXiv Detail & Related papers (2020-05-06T07:41:22Z) - Rethinking Differentiable Search for Mixed-Precision Neural Networks [83.55785779504868]
Low-precision networks with weights and activations quantized to low bit-width are widely used to accelerate inference on edge devices.
Current solutions are uniform, using identical bit-width for all filters.
This fails to account for the different sensitivities of different filters and is suboptimal.
Mixed-precision networks address this problem, by tuning the bit-width to individual filter requirements.
arXiv Detail & Related papers (2020-04-13T07:02:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.