Methods for Pruning Deep Neural Networks

• URL: http://arxiv.org/abs/2011.00241v2
• Date: Fri, 30 Jul 2021 16:36:00 GMT
• Title: Methods for Pruning Deep Neural Networks
• Authors: Sunil Vadera and Salem Ameen
• Abstract summary: This paper presents a survey of methods for pruning deep neural networks. It begins by categorising over 150 studies based on the underlying approach used. It focuses on three categories: methods that use magnitude based pruning, methods that utilise clustering to identify redundancy, and methods that use sensitivity analysis.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a survey of methods for pruning deep neural networks. It begins by categorising over 150 studies based on the underlying approach used and then focuses on three categories: methods that use magnitude based pruning, methods that utilise clustering to identify redundancy, and methods that use sensitivity analysis to assess the effect of pruning. Some of the key influencing studies within these categories are presented to highlight the underlying approaches and results achieved. Most studies present results which are distributed in the literature as new architectures, algorithms and data sets have developed with time, making comparison across different studied difficult. The paper therefore provides a resource for the community that can be used to quickly compare the results from many different methods on a variety of data sets, and a range of architectures, including AlexNet, ResNet, DenseNet and VGG. The resource is illustrated by comparing the results published for pruning AlexNet and ResNet50 on ImageNet and ResNet56 and VGG16 on the CIFAR10 data to reveal which pruning methods work well in terms of retaining accuracy whilst achieving good compression rates. The paper concludes by identifying some promising directions for future research.

Related papers

• Block Pruning for Enhanced Efficiency in Convolutional Neural Networks [7.110116320545541]
  This paper presents a novel approach to network pruning, targeting block pruning in deep neural networks for edge computing environments. Our method diverges from traditional techniques that utilize proxy metrics, instead employing a direct block removal strategy to assess the impact on classification accuracy.
  arXiv  Detail & Related papers  (2023-12-28T08:54:48Z)
• A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations [20.958265043544603]
  Modern deep neural networks come with massive model sizes that require significant computational and storage resources. Researchers have increasingly explored pruning techniques as a popular research direction in neural network compression. We provide a review of existing research works on deep neural network pruning in a taxonomy of 1) universal/specific speedup, 2) when to prune, 3) how to prune, and 4) fusion of pruning and other compression techniques.
  arXiv  Detail & Related papers  (2023-08-13T13:34:04Z)
• Advancing 3D finger knuckle recognition via deep feature learning [51.871256510747465]
  Contactless 3D finger knuckle patterns have emerged as an effective biometric identifier due to its discriminativeness, visibility from a distance, and convenience. Recent research has developed a deep feature collaboration network which simultaneously incorporates intermediate features from deep neural networks with multiple scales. This paper advances this approach by investigating the possibility of learning a discriminative feature vector with the least possible dimension for representing 3D finger knuckle images.
  arXiv  Detail & Related papers  (2023-01-07T20:55:16Z)
• How to Train Your Neural Network: A Comparative Evaluation [1.3654846342364304]
  We discuss and compare current state-of-the-art frameworks for large scale distributed deep learning. We present empirical results comparing their performance on large image and language training tasks. Based on our results, we discuss algorithmic and implementation portions of each framework which hinder performance.
  arXiv  Detail & Related papers  (2021-11-09T04:24:42Z)
• CvS: Classification via Segmentation For Small Datasets [52.821178654631254]
  This paper presents CvS, a cost-effective classifier for small datasets that derives the classification labels from predicting the segmentation maps. We evaluate the effectiveness of our framework on diverse problems showing that CvS is able to achieve much higher classification results compared to previous methods when given only a handful of examples.
  arXiv  Detail & Related papers  (2021-10-29T18:41:15Z)
• Robust Pooling through the Data Mode [5.7564383437854625]
  This paper proposes a novel deep learning solution that includes a novel robust pooling layer. The proposed pooling layer looks for data a mode/cluster using two methods, RANSAC, and histogram, as clusters are indicative of models. We tested the pooling layer into frameworks such as Point-based and graph-based neural networks, and the tests showed enhanced robustness as compared to robust state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-21T04:35:24Z)
• A SAR speckle filter based on Residual Convolutional Neural Networks [68.8204255655161]
  This work aims to present a novel method for filtering the speckle noise from Sentinel-1 data by applying Deep Learning (DL) algorithms, based on Convolutional Neural Networks (CNNs) The obtained results, if compared with the state of the art, show a clear improvement in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM)
  arXiv  Detail & Related papers  (2021-04-19T14:43:07Z)
• Manifold Regularized Dynamic Network Pruning [102.24146031250034]
  This paper proposes a new paradigm that dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the proposed method is verified on several benchmarks, which shows better performance in terms of both accuracy and computational cost.
  arXiv  Detail & Related papers  (2021-03-10T03:59:03Z)
• Learning Robust Feature Representations for Scene Text Detection [0.0]
  We present a network architecture derived from the loss to maximize conditional log-likelihood. By extending the layer of latent variables to multiple layers, the network is able to learn robust features on scale. In experiments, the proposed algorithm significantly outperforms state-of-the-art methods in terms of both recall and precision.
  arXiv  Detail & Related papers  (2020-05-26T01:06:47Z)
• Deep Learning for Community Detection: Progress, Challenges and Opportunities [79.26787486888549]
  Article summarizes the contributions of the various frameworks, models, and algorithms in deep neural networks. This article summarizes the contributions of the various frameworks, models, and algorithms in deep neural networks.
  arXiv  Detail & Related papers  (2020-05-17T11:22:11Z)
• Attentive CutMix: An Enhanced Data Augmentation Approach for Deep Learning Based Image Classification [58.20132466198622]
  We propose Attentive CutMix, a naturally enhanced augmentation strategy based on CutMix. In each training iteration, we choose the most descriptive regions based on the intermediate attention maps from a feature extractor. Our proposed method is simple yet effective, easy to implement and can boost the baseline significantly.
  arXiv  Detail & Related papers  (2020-03-29T15:01:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.