Novel Adaptive Binary Search Strategy-First Hybrid Pyramid- and Clustering-Based CNN Filter Pruning Method without Parameters Setting

• URL: http://arxiv.org/abs/2006.04451v2
• Date: Fri, 30 Apr 2021 07:33:00 GMT
• Title: Novel Adaptive Binary Search Strategy-First Hybrid Pyramid- and Clustering-Based CNN Filter Pruning Method without Parameters Setting
• Authors: Kuo-Liang Chung, Yu-Lun Chang, and Bo-Wei Tsai
• Abstract summary: Pruning redundant filters in CNN models has received growing attention. We propose an adaptive binary search-first hybrid pyramid- and clustering-based (ABS HPC) method for pruning filters automatically. Based on the practical dataset and the CNN models, with higher accuracy, the thorough experimental results demonstrated the significant parameters and floating-point operations reduction merits of the proposed filter pruning method.
• Score: 3.7468898363447654
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pruning redundant filters in CNN models has received growing attention. In this paper, we propose an adaptive binary search-first hybrid pyramid- and clustering-based (ABSHPC-based) method for pruning filters automatically. In our method, for each convolutional layer, initially a hybrid pyramid data structure is constructed to store the hierarchical information of each filter. Given a tolerant accuracy loss, without parameters setting, we begin from the last convolutional layer to the first layer; for each considered layer with less or equal pruning rate relative to its previous layer, our ABSHPC-based process is applied to optimally partition all filters to clusters, where each cluster is thus represented by the filter with the median root mean of the hybrid pyramid, leading to maximal removal of redundant filters. Based on the practical dataset and the CNN models, with higher accuracy, the thorough experimental results demonstrated the significant parameters and floating-point operations reduction merits of the proposed filter pruning method relative to the state-of-the-art methods.

Related papers

• A Greedy Hierarchical Approach to Whole-Network Filter-Pruning in CNNs [2.188091591747149]
  Whole-network filter pruning algorithms prune varying fractions of filters from each layer, hence providing greater flexibility. This paper proposes a two-level hierarchical approach for whole-network filter pruning which is efficient and uses the classification loss as the final criterion. Our method reduces the RAM requirement for ResNext101 from 7.6 GB to 1.5 GB and achieves a 94% reduction in FLOPS without losing accuracy on CIFAR-10.
  arXiv  Detail & Related papers  (2024-08-22T03:59:57Z)
• Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler [103.97487121678276]
  Filter pruning simultaneously accelerates the computation and reduces the memory overhead of CNNs. We propose a novel Knowledge-driven Differential Filter Sampler(KDFS) with Masked Filter Modeling(MFM) framework for filter pruning.
  arXiv  Detail & Related papers  (2023-07-01T02:28:41Z)
• Focus Your Attention (with Adaptive IIR Filters) [62.80628327613344]
  We present a new layer in which dynamic (i.e.,input-dependent) Infinite Impulse Response (IIR) filters of order two are used to process the input sequence. Despite their relatively low order, the causal adaptive filters are shown to focus attention on the relevant sequence elements.
  arXiv  Detail & Related papers  (2023-05-24T09:42:30Z)
• Filter Pruning via Filters Similarity in Consecutive Layers [20.29555787754269]
  Filter pruning is widely adopted to compress and accelerate the Convolutional Neural Networks (CNNs) We intuitively propose a novel pruning method by explicitly leveraging the Filters Similarity in Consecutive Layers (FSCL) Experiments demonstrate the effectiveness of FSCL, and it yields remarkable improvement over state-of-the-art on accuracy, FLOPs and parameter reduction.
  arXiv  Detail & Related papers  (2023-04-26T09:18:38Z)
• Maximum margin learning of t-SPNs for cell classification with filtered input [19.66983830788521]
  The t-SPN architecture is learned by maximizing the margin. L2-regularization (REG) is considered along with the maximum margin (MM) criterion in the learning process. On both HEp-2 and Feulgen benchmark datasets, the t-SPN architecture learned based on the max-margin criterion with regularization produced the highest accuracy rate.
  arXiv  Detail & Related papers  (2023-03-16T03:45:46Z)
• Asymptotic Soft Cluster Pruning for Deep Neural Networks [5.311178623385279]
  Filter pruning method introduces structural sparsity by removing selected filters. We propose a novel filter pruning method called Asymptotic Soft Cluster Pruning. Our method can achieve competitive results compared with many state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2022-06-16T13:58:58Z)
• Unsharp Mask Guided Filtering [53.14430987860308]
  The goal of this paper is guided image filtering, which emphasizes the importance of structure transfer during filtering. We propose a new and simplified formulation of the guided filter inspired by unsharp masking. Our formulation enjoys a filtering prior to a low-pass filter and enables explicit structure transfer by estimating a single coefficient.
  arXiv  Detail & Related papers  (2021-06-02T19:15:34Z)
• Deep Shells: Unsupervised Shape Correspondence with Optimal Transport [52.646396621449]
  We propose a novel unsupervised learning approach to 3D shape correspondence. We show that the proposed method significantly improves over the state-of-the-art on multiple datasets.
  arXiv  Detail & Related papers  (2020-10-28T22:24:07Z)
• Dependency Aware Filter Pruning [74.69495455411987]
  Pruning a proportion of unimportant filters is an efficient way to mitigate the inference cost. Previous work prunes filters according to their weight norms or the corresponding batch-norm scaling factors. We propose a novel mechanism to dynamically control the sparsity-inducing regularization so as to achieve the desired sparsity.
  arXiv  Detail & Related papers  (2020-05-06T07:41:22Z)
• MINT: Deep Network Compression via Mutual Information-based Neuron Trimming [32.449324736645586]
  Mutual Information-based Neuron Trimming (MINT) approaches deep compression via pruning. MINT enforces sparsity based on the strength of the relationship between filters of adjacent layers. When pruning a network, we ensure that retained filters contribute the majority of the information towards succeeding layers.
  arXiv  Detail & Related papers  (2020-03-18T21:05:02Z)
• Clustering Binary Data by Application of Combinatorial Optimization Heuristics [52.77024349608834]
  We study clustering methods for binary data, first defining aggregation criteria that measure the compactness of clusters. Five new and original methods are introduced, using neighborhoods and population behavior optimization metaheuristics. From a set of 16 data tables generated by a quasi-Monte Carlo experiment, a comparison is performed for one of the aggregations using L1 dissimilarity, with hierarchical clustering, and a version of k-means: partitioning around medoids or PAM.
  arXiv  Detail & Related papers  (2020-01-06T23:33:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.