Compressing audio CNNs with graph centrality based filter pruning

• URL: http://arxiv.org/abs/2305.03391v1
• Date: Fri, 5 May 2023 09:38:05 GMT
• Title: Compressing audio CNNs with graph centrality based filter pruning
• Authors: James A King, Arshdeep Singh, Mark D. Plumbley
• Abstract summary: Convolutional neural networks (CNNs) are commonplace in high-performing solutions to many real-world problems. CNNs have many parameters and filters, with some having a larger impact on the performance than others. We propose a pruning framework that eliminates filters with the highest "commonality"
• Score: 20.028643659869573
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolutional neural networks (CNNs) are commonplace in high-performing solutions to many real-world problems, such as audio classification. CNNs have many parameters and filters, with some having a larger impact on the performance than others. This means that networks may contain many unnecessary filters, increasing a CNN's computation and memory requirements while providing limited performance benefits. To make CNNs more efficient, we propose a pruning framework that eliminates filters with the highest "commonality". We measure this commonality using the graph-theoretic concept of "centrality". We hypothesise that a filter with a high centrality should be eliminated as it represents commonality and can be replaced by other filters without affecting the performance of a network much. An experimental evaluation of the proposed framework is performed on acoustic scene classification and audio tagging. On the DCASE 2021 Task 1A baseline network, our proposed method reduces computations per inference by 71\% with 50\% fewer parameters at less than a two percentage point drop in accuracy compared to the original network. For large-scale CNNs such as PANNs designed for audio tagging, our method reduces 24\% computations per inference with 41\% fewer parameters at a slight improvement in performance.

Related papers

• Attention-based Feature Compression for CNN Inference Offloading in Edge Computing [93.67044879636093]
  This paper studies the computational offloading of CNN inference in device-edge co-inference systems. We propose a novel autoencoder-based CNN architecture (AECNN) for effective feature extraction at end-device. Experiments show that AECNN can compress the intermediate data by more than 256x with only about 4% accuracy loss.
  arXiv  Detail & Related papers  (2022-11-24T18:10:01Z)
• Simple Pooling Front-ends For Efficient Audio Classification [56.59107110017436]
  We show that eliminating the temporal redundancy in the input audio features could be an effective approach for efficient audio classification. We propose a family of simple pooling front-ends (SimPFs) which use simple non-parametric pooling operations to reduce the redundant information. SimPFs can achieve a reduction in more than half of the number of floating point operations for off-the-shelf audio neural networks.
  arXiv  Detail & Related papers  (2022-10-03T14:00:41Z)
• Low-complexity CNNs for Acoustic Scene Classification [23.661189257759535]
  This paper presents a low-complexity framework for acoustic scene classification (ASC) Most of the frameworks designed for ASC use convolutional neural networks (CNNs) due to their learning ability and improved performance. CNNs are resource hungry due to their large size and high computational complexity.
  arXiv  Detail & Related papers  (2022-07-23T14:37:39Z)
• A Passive Similarity based CNN Filter Pruning for Efficient Acoustic Scene Classification [23.661189257759535]
  We present a method to develop low-complexity convolutional neural networks (CNNs) for acoustic scene classification (ASC) We propose a passive filter pruning framework, where a few convolutional filters from the CNNs are eliminated to yield compressed CNNs. The proposed method is simple, reduces computations per inference by 27%, with 25% fewer parameters, with less than 1% drop in accuracy.
  arXiv  Detail & Related papers  (2022-03-29T17:00:06Z)
• Batch Normalization Tells You Which Filter is Important [49.903610684578716]
  We propose a simple yet effective filter pruning method by evaluating the importance of each filter based on the BN parameters of pre-trained CNNs. The experimental results on CIFAR-10 and ImageNet demonstrate that the proposed method can achieve outstanding performance.
  arXiv  Detail & Related papers  (2021-12-02T12:04:59Z)
• Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion [139.71852076031962]
  We present a novel filter pruning method, dubbed dynamic-coded filter fusion (DCFF) We derive compact CNNs in a computation-economical and regularization-free manner for efficient image classification. Our DCFF derives a compact VGGNet-16 with only 72.77M FLOPs and 1.06M parameters while reaching top-1 accuracy of 93.47%.
  arXiv  Detail & Related papers  (2021-07-14T18:07:38Z)
• Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms [0.9954382983583577]
  We propose a method to optimally quantize the weights, biases and activations of each layer of a pre-trained CNN. We find that layer-wise quantization of parameters significantly helps in this process.
  arXiv  Detail & Related papers  (2021-02-03T17:05:55Z)
• Rescaling CNN through Learnable Repetition of Network Parameters [2.137666194897132]
  We present a novel rescaling strategy for CNNs based on learnable repetition of its parameters. We show that small base networks when rescaled, can provide performance comparable to deeper networks with as low as 6% of optimization parameters of the deeper one.
  arXiv  Detail & Related papers  (2021-01-14T15:03:25Z)
• Filter Grafting for Deep Neural Networks: Reason, Method, and Cultivation [86.91324735966766]
  Filter is the key component in modern convolutional neural networks (CNNs) In this paper, we introduce filter grafting (textbfMethod) to achieve this goal. We develop a novel criterion to measure the information of filters and an adaptive weighting strategy to balance the grafted information among networks.
  arXiv  Detail & Related papers  (2020-04-26T08:36:26Z)
• Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks [52.972605601174955]
  We show a ResNet-type CNN can attain the minimax optimal error rates in important function classes. We derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes.
  arXiv  Detail & Related papers  (2019-03-24T19:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.