Layer-adaptive Structured Pruning Guided by Latency

• URL: http://arxiv.org/abs/2305.14403v1
• Date: Tue, 23 May 2023 11:18:37 GMT
• Title: Layer-adaptive Structured Pruning Guided by Latency
• Authors: Siyuan Pan, Linna Zhang, Jie Zhang, Xiaoshuang Li, Liang Hou, Xiaobing Tu
• Abstract summary: Structured pruning can simplify network architecture and improve inference speed. We propose a global importance score SP-LAMP by deriving a global importance score LAMP from unstructured pruning to structured pruning. Experimental results in ResNet56 on CIFAR10 demonstrate that our algorithm achieves lower latency compared to alternative approaches.
• Score: 7.193554978191659
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured pruning can simplify network architecture and improve inference speed. Combined with the underlying hardware and inference engine in which the final model is deployed, better results can be obtained by using latency collaborative loss function to guide network pruning together. Existing pruning methods that optimize latency have demonstrated leading performance, however, they often overlook the hardware features and connection in the network. To address this problem, we propose a global importance score SP-LAMP(Structured Pruning Layer-Adaptive Magnitude-based Pruning) by deriving a global importance score LAMP from unstructured pruning to structured pruning. In SP-LAMP, each layer includes a filter with an SP-LAMP score of 1, and the remaining filters are grouped. We utilize a group knapsack solver to maximize the SP-LAMP score under latency constraints. In addition, we improve the strategy of collect the latency to make it more accurate. In particular, for ResNet50/ResNet18 on ImageNet and CIFAR10, SP-LAMP is 1.28x/8.45x faster with +1.7%/-1.57% top-1 accuracy changed, respectively. Experimental results in ResNet56 on CIFAR10 demonstrate that our algorithm achieves lower latency compared to alternative approaches while ensuring accuracy and FLOPs.

Related papers

• Latency-aware Unified Dynamic Networks for Efficient Image Recognition [72.8951331472913]
  LAUDNet is a framework to bridge the theoretical and practical efficiency gap in dynamic networks. It integrates three primary dynamic paradigms-spatially adaptive computation, dynamic layer skipping, and dynamic channel skipping. It can notably reduce the latency of models like ResNet by over 50% on platforms such as V100,3090, and TX2 GPUs.
  arXiv  Detail & Related papers  (2023-08-30T10:57:41Z)
• DeepReShape: Redesigning Neural Networks for Efficient Private Inference [3.7802450241986945]
  Recent work has shown that FLOPs for PI can no longer be ignored and incur high latency penalties. We develop DeepReShape, a technique that optimize neural network architectures under PI's constraints.
  arXiv  Detail & Related papers  (2023-04-20T18:27:02Z)
• Structural Pruning via Latency-Saliency Knapsack [40.562285600570924]
  Hardware-Aware Structural Pruning (HALP) HALP formulates structural pruning as a global resource allocation optimization problem. Uses latency lookup table to track latency reduction potential and global saliency score to gauge accuracy drop.
  arXiv  Detail & Related papers  (2022-10-13T01:41:59Z)
• End-to-End Sensitivity-Based Filter Pruning [49.61707925611295]
  We present a sensitivity-based filter pruning algorithm (SbF-Pruner) to learn the importance scores of filters of each layer end-to-end. Our method learns the scores from the filter weights, enabling it to account for the correlations between the filters of each layer.
  arXiv  Detail & Related papers  (2022-04-15T10:21:05Z)
• Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration [71.80326738527734]
  We propose a general, fine-grained structured pruning scheme and corresponding compiler optimizations. We show that our pruning scheme mapping methods, together with the general fine-grained structured pruning scheme, outperform the state-of-the-art DNN optimization framework.
  arXiv  Detail & Related papers  (2021-11-22T23:53:14Z)
• HALP: Hardware-Aware Latency Pruning [25.071902504529465]
  Hardware-Aware Structural Pruning (HALP) HALP formulates structural pruning as a global resource allocation optimization problem. We examine HALP on both classification and detection tasks, over varying networks, on ImageNet and VOC datasets.
  arXiv  Detail & Related papers  (2021-10-20T22:34:51Z)
• HANT: Hardware-Aware Network Transformation [82.54824188745887]
  We propose hardware-aware network transformation (HANT) HANT replaces inefficient operations with more efficient alternatives using a neural architecture search like approach. Our results on accelerating the EfficientNet family show that HANT can accelerate them by up to 3.6x with 0.4% drop in the top-1 accuracy on the ImageNet dataset.
  arXiv  Detail & Related papers  (2021-07-12T18:46:34Z)
• Efficient Incorporation of Multiple Latency Targets in the Once-For-All Network [0.0]
  We introduce two strategies that use warm starting and randomized network pruning for the efficient incorporation of multiple latency targets in the OFA network. We evaluate these strategies against the current OFA implementation and demonstrate that our strategies offer significant running time performance gains.
  arXiv  Detail & Related papers  (2020-12-12T07:34:09Z)
• Weight-dependent Gates for Network Pruning [24.795174721078528]
  This paper argues that the pruning decision should depend on the convolutional weights, and thus proposes novel weight-dependent gates (W-Gates) to learn the information from filter weights and obtain binary gates to prune or keep the filters automatically. We have demonstrated the effectiveness of the proposed method on ResNet34, ResNet50, and MobileNet V2, respectively achieving up to 1.33/1.28/1.1 higher Top-1 accuracy with lower hardware latency on ImageNet.
  arXiv  Detail & Related papers  (2020-07-04T10:29:07Z)
• Toward fast and accurate human pose estimation via soft-gated skip connections [97.06882200076096]
  This paper is on highly accurate and highly efficient human pose estimation. We re-analyze this design choice in the context of improving both the accuracy and the efficiency over the state-of-the-art. Our model achieves state-of-the-art results on the MPII and LSP datasets.
  arXiv  Detail & Related papers  (2020-02-25T18:51:51Z)
• Convolutional Networks with Dense Connectivity [59.30634544498946]
  We introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion. For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. We evaluate our proposed architecture on four highly competitive object recognition benchmark tasks.
  arXiv  Detail & Related papers  (2020-01-08T06:54:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.