Abstract: As the convolutional neural network (CNN) gets deeper and wider in recent
years, the requirements for the amount of data and hardware resources have
gradually increased. Meanwhile, CNN also reveals salient redundancy in several
tasks. The existing magnitude-based pruning methods are efficient, but the
performance of the compressed network is unpredictable. While the accuracy loss
after pruning based on the structure sensitivity is relatively slight, the
process is time-consuming and the algorithm complexity is notable. In this
article, we propose a novel automatic channel pruning method (ACP).
Specifically, we firstly perform layer-wise channel clustering via the
similarity of the feature maps to perform preliminary pruning on the network.
Then a population initialization method is introduced to transform the pruned
structure into a candidate population. Finally, we conduct searching and
optimizing iteratively based on the particle swarm optimization (PSO) to find
the optimal compressed structure. The compact network is then retrained to
mitigate the accuracy loss from pruning. Our method is evaluated against
several state-of-the-art CNNs on three different classification datasets
CIFAR-10/100 and ILSVRC-2012. On the ILSVRC-2012, when removing 64.36%
parameters and 63.34% floating-point operations (FLOPs) of ResNet-50, the Top-1
and Top-5 accuracy drop are less than 0.9%. Moreover, we demonstrate that
without harming overall performance it is possible to compress SSD by more than
50% on the target detection dataset PASCAL VOC. It further verifies that the
proposed method can also be applied to other CNNs and application scenarios.