Modeling of Pruning Techniques for Deep Neural Networks Simplification

• URL: http://arxiv.org/abs/2001.04062v1
• Date: Mon, 13 Jan 2020 04:51:59 GMT
• Title: Modeling of Pruning Techniques for Deep Neural Networks Simplification
• Authors: Morteza Mousa Pasandi, Mohsen Hajabdollahi, Nader Karimi, Shadrokh Samavi
• Abstract summary: Convolutional Neural Networks (CNNs) suffer from different issues, such as computational complexity and the number of parameters. In recent years pruning techniques are employed to reduce the number of operations and model size in CNNs. Various techniques and tricks accompany pruning methods, and there is not a unifying framework to model all the pruning methods.
• Score: 8.75217589103206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional Neural Networks (CNNs) suffer from different issues, such as computational complexity and the number of parameters. In recent years pruning techniques are employed to reduce the number of operations and model size in CNNs. Different pruning methods are proposed, which are based on pruning the connections, channels, and filters. Various techniques and tricks accompany pruning methods, and there is not a unifying framework to model all the pruning methods. In this paper pruning methods are investigated, and a general model which is contained the majority of pruning techniques is proposed. The advantages and disadvantages of the pruning methods can be identified, and all of them can be summarized under this model. The final goal of this model is to provide a general approach for all of the pruning methods with different structures and applications.

Related papers

• Structurally Prune Anything: Any Architecture, Any Framework, Any Time [84.6210631783801]
  We introduce Structurally Prune Anything (SPA), a versatile structured pruning framework for neural networks. SPA supports pruning at any time, either before training, after training with fine-tuning, or after training without fine-tuning. In extensive experiments, SPA shows competitive to state-of-the-art pruning performance across various architectures.
  arXiv  Detail & Related papers  (2024-03-03T13:49:49Z)
• DepGraph: Towards Any Structural Pruning [68.40343338847664]
  We study general structural pruning of arbitrary architecture like CNNs, RNNs, GNNs and Transformers. We propose a general and fully automatic method, emphDependency Graph (DepGraph), to explicitly model the dependency between layers and comprehensively group parameters for pruning. In this work, we extensively evaluate our method on several architectures and tasks, including ResNe(X)t, DenseNet, MobileNet and Vision transformer for images, GAT for graph, DGCNN for 3D point cloud, alongside LSTM for language, and demonstrate that, even with a
  arXiv  Detail & Related papers  (2023-01-30T14:02:33Z)
• Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps [85.49020931411825]
  Convolutional Neural Networks (CNNs) compression is crucial to deploying these models in edge devices with limited resources. We propose to address the channel pruning problem from a novel perspective by leveraging the interpretations of a model to steer the pruning process. We tackle this challenge by introducing a selector model that predicts real-time smooth saliency masks for pruned models.
  arXiv  Detail & Related papers  (2022-09-07T01:12:11Z)
• Deep Neural Networks pruning via the Structured Perspective Regularization [5.061851539114448]
  In Machine Learning, Artificial Neural Networks (ANNs) are a very powerful tool, broadly used in many applications. One of the most popular compression approaches is emphpruning, whereby entire elements of the ANN (links, nodes, channels, ldots) and the corresponding weights are deleted. Since the nature of the problem is inherently (what elements to prune and what not), we propose a new pruning method based on Operational Research tools.
  arXiv  Detail & Related papers  (2022-06-28T14:58:51Z)
• Automatic Block-wise Pruning with Auxiliary Gating Structures for Deep Convolutional Neural Networks [9.293334856614628]
  This paper presents a novel structured network pruning method with auxiliary gating structures. Our experiments demonstrate that our method can achieve state-of-the-arts compression performance for the classification tasks.
  arXiv  Detail & Related papers  (2022-05-07T09:03:32Z)
• Emerging Paradigms of Neural Network Pruning [82.9322109208353]
  Pruning is adopted as a post-processing solution to this problem, which aims to remove unnecessary parameters in a neural network with little performance compromised. Recent works challenge this belief by discovering random sparse networks which can be trained to match the performance with their dense counterpart. This survey seeks to bridge the gap by proposing a general pruning framework so that the emerging pruning paradigms can be accommodated well with the traditional one.
  arXiv  Detail & Related papers  (2021-03-11T05:01:52Z)
• Manifold Regularized Dynamic Network Pruning [102.24146031250034]
  This paper proposes a new paradigm that dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the proposed method is verified on several benchmarks, which shows better performance in terms of both accuracy and computational cost.
  arXiv  Detail & Related papers  (2021-03-10T03:59:03Z)
• A Framework for Neural Network Pruning Using Gibbs Distributions [34.0576955010317]
  Gibbs pruning is a novel framework for expressing and designing neural network pruning methods. It can train and prune a network simultaneously in such a way that the learned weights and pruning mask are well-adapted for each other. We achieve a new state-of-the-art result for pruning ResNet-56 with the CIFAR-10 dataset.
  arXiv  Detail & Related papers  (2020-06-08T23:04:53Z)
• Deep Unfolding Network for Image Super-Resolution [159.50726840791697]
  This paper proposes an end-to-end trainable unfolding network which leverages both learning-based methods and model-based methods. The proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model.
  arXiv  Detail & Related papers  (2020-03-23T17:55:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.