PAODING: A High-fidelity Data-free Pruning Toolkit for Debloating Pre-trained Neural Networks

• URL: http://arxiv.org/abs/2405.00074v1
• Date: Tue, 30 Apr 2024 07:24:41 GMT
• Title: PAODING: A High-fidelity Data-free Pruning Toolkit for Debloating Pre-trained Neural Networks
• Authors: Mark Huasong Meng, Hao Guan, Liuhuo Wan, Sin Gee Teo, Guangdong Bai, Jin Song Dong,
• Abstract summary: PAODING is a toolkit to debloat pretrained neural network models through the lens of data-free pruning. It can significantly reduce the model size and generalize on different datasets and models. It can also preserve the model fidelity in terms of test accuracy and adversarial robustness.
• Score: 11.600305034972996
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present PAODING, a toolkit to debloat pretrained neural network models through the lens of data-free pruning. To preserve the model fidelity, PAODING adopts an iterative process, which dynamically measures the effect of deleting a neuron to identify candidates that have the least impact to the output layer. Our evaluation shows that PAODING can significantly reduce the model size, generalize on different datasets and models, and meanwhile preserve the model fidelity in terms of test accuracy and adversarial robustness. PAODING is publicly available on PyPI via https://pypi.org/project/paoding-dl.

Related papers

• Watermarking for Out-of-distribution Detection [76.20630986010114]
  Out-of-distribution (OOD) detection aims to identify OOD data based on representations extracted from well-trained deep models. We propose a general methodology named watermarking in this paper. We learn a unified pattern that is superimposed onto features of original data, and the model's detection capability is largely boosted after watermarking.
  arXiv  Detail & Related papers  (2022-10-27T06:12:32Z)
• Adversarial Robustness Assessment of NeuroEvolution Approaches [1.237556184089774]
  We evaluate the robustness of models found by two NeuroEvolution approaches on the CIFAR-10 image classification task. Our results show that when the evolved models are attacked with iterative methods, their accuracy usually drops to, or close to, zero. Some of these techniques can exacerbate the perturbations added to the original inputs, potentially harming robustness.
  arXiv  Detail & Related papers  (2022-07-12T10:40:19Z)
• Paoding: Supervised Robustness-preserving Data-free Neural Network Pruning [3.6953655494795776]
  We study the neural network pruning in the emphdata-free context. We replace the traditional aggressive one-shot strategy with a conservative one that treats the pruning as a progressive process. Our method is implemented as a Python package named textscPaoding and evaluated with a series of experiments on diverse neural network models.
  arXiv  Detail & Related papers  (2022-04-02T07:09:17Z)
• EARLIN: Early Out-of-Distribution Detection for Resource-efficient Collaborative Inference [4.826988182025783]
  Collaborative inference enables resource-constrained edge devices to make inferences by uploading inputs to a server. While this setup works cost-effectively for successful inferences, it severely underperforms when the model faces input samples on which the model was not trained. We propose a novel lightweight OOD detection approach that mines important features from the shallow layers of a pretrained CNN model.
  arXiv  Detail & Related papers  (2021-06-25T18:43:23Z)
• Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
  We show that pruning improves generalization for neural ODEs in generative modeling. We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
  arXiv  Detail & Related papers  (2021-06-24T01:40:17Z)
• Contextual Dropout: An Efficient Sample-Dependent Dropout Module [60.63525456640462]
  Dropout has been demonstrated as a simple and effective module to regularize the training process of deep neural networks. We propose contextual dropout with an efficient structural design as a simple and scalable sample-dependent dropout module. Our experimental results show that the proposed method outperforms baseline methods in terms of both accuracy and quality of uncertainty estimation.
  arXiv  Detail & Related papers  (2021-03-06T19:30:32Z)
• Firearm Detection via Convolutional Neural Networks: Comparing a Semantic Segmentation Model Against End-to-End Solutions [68.8204255655161]
  Threat detection of weapons and aggressive behavior from live video can be used for rapid detection and prevention of potentially deadly incidents. One way for achieving this is through the use of artificial intelligence and, in particular, machine learning for image analysis. We compare a traditional monolithic end-to-end deep learning model and a previously proposed model based on an ensemble of simpler neural networks detecting fire-weapons via semantic segmentation.
  arXiv  Detail & Related papers  (2020-12-17T15:19:29Z)
• Dynamic Model Pruning with Feedback [64.019079257231]
  We propose a novel model compression method that generates a sparse trained model without additional overhead. We evaluate our method on CIFAR-10 and ImageNet, and show that the obtained sparse models can reach the state-of-the-art performance of dense models.
  arXiv  Detail & Related papers  (2020-06-12T15:07:08Z)
• Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems [83.98774574197613]
  We take one of the simplest inference methods, a truncated max-product Belief propagation, and add what is necessary to make it a proper component of a deep learning model. This BP-Layer can be used as the final or an intermediate block in convolutional neural networks (CNNs) The model is applicable to a range of dense prediction problems, is well-trainable and provides parameter-efficient and robust solutions in stereo, optical flow and semantic segmentation.
  arXiv  Detail & Related papers  (2020-03-13T13:11:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.