Abstract: Deep Convolutional Neural Networks (DCNNs) have shown promising results in
several visual recognition problems which motivated the researchers to propose
popular architectures such as LeNet, AlexNet, VGGNet, ResNet, and many more.
These architectures come at a cost of high computational complexity and
parameter storage. To get rid of storage and computational complexity, deep
model compression methods have been evolved. We propose a novel History Based
Filter Pruning (HBFP) method that utilizes network training history for filter
pruning. Specifically, we prune the redundant filters by observing similar
patterns in the L1-norms of filters (absolute sum of weights) over the training
epochs. We iteratively prune the redundant filters of a CNN in three steps.
First, we train the model and select the filter pairs with redundant filters in
each pair. Next, we optimize the network to increase the similarity between the
filters in a pair. It facilitates us to prune one filter from each pair based
on its importance without much information loss. Finally, we retrain the
network to regain the performance, which is dropped due to filter pruning. We
test our approach on popular architectures such as LeNet-5 on MNIST dataset and
VGG-16, ResNet-56, and ResNet-110 on CIFAR-10 dataset. The proposed pruning
method outperforms the state-of-the-art in terms of FLOPs reduction
(floating-point operations) by 97.98%, 83.42%, 78.43%, and 74.95% for LeNet-5,
VGG-16, ResNet-56, and ResNet-110 models, respectively, while maintaining the
less error rate.