AutoDFP: Automatic Data-Free Pruning via Channel Similarity Reconstruction

• URL: http://arxiv.org/abs/2403.08204v1
• Date: Wed, 13 Mar 2024 02:56:31 GMT
• Title: AutoDFP: Automatic Data-Free Pruning via Channel Similarity Reconstruction
• Authors: Siqi Li, Jun Chen, Jingyang Xiang, Chengrui Zhu, Yong Liu
• Abstract summary: We propose the Automatic Data-Free Pruning (AutoDFP) method that achieves automatic pruning and reconstruction without fine-tuning. We evaluate AutoDFP with multiple networks on multiple datasets, achieving impressive compression results.
• Score: 18.589013910402237
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured pruning methods are developed to bridge the gap between the massive scale of neural networks and the limited hardware resources. Most current structured pruning methods rely on training datasets to fine-tune the compressed model, resulting in high computational burdens and being inapplicable for scenarios with stringent requirements on privacy and security. As an alternative, some data-free methods have been proposed, however, these methods often require handcraft parameter tuning and can only achieve inflexible reconstruction. In this paper, we propose the Automatic Data-Free Pruning (AutoDFP) method that achieves automatic pruning and reconstruction without fine-tuning. Our approach is based on the assumption that the loss of information can be partially compensated by retaining focused information from similar channels. Specifically, We formulate data-free pruning as an optimization problem, which can be effectively addressed through reinforcement learning. AutoDFP assesses the similarity of channels for each layer and provides this information to the reinforcement learning agent, guiding the pruning and reconstruction process of the network. We evaluate AutoDFP with multiple networks on multiple datasets, achieving impressive compression results. For instance, on the CIFAR-10 dataset, AutoDFP demonstrates a 2.87\% reduction in accuracy loss compared to the recently proposed data-free pruning method DFPC with fewer FLOPs on VGG-16. Furthermore, on the ImageNet dataset, AutoDFP achieves 43.17\% higher accuracy than the SOTA method with the same 80\% preserved ratio on MobileNet-V1.

Related papers

• Adaptive Structured Pruning of Convolutional Neural Networks for Time Series Classification [0.776514389034479]
  We propose Dynamic Structured Pruning (DSP), a fully automatic, structured pruning framework for convolution-based TSC models.<n>Our approach achieves an average compression of 58% for LITETime and 75% for InceptionTime architectures while maintaining classification accuracy.<n>Redundancy analyses confirm that DSP produces compact and informative representations, offering a practical path for scalable and efficient deep TSC deployment.
  arXiv  Detail & Related papers  (2026-02-13T09:18:59Z)
• Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning [71.30276778807068]
  We propose a unified framework that strategically coordinates sample pruning and token pruning.<n>Q-Tuning achieves a +38% average improvement over the full-data SFT baseline using only 12.5% of the original training data.
  arXiv  Detail & Related papers  (2025-09-28T13:27:38Z)
• Flexible Automatic Identification and Removal (FAIR)-Pruner: An Efficient Neural Network Pruning Method [11.575879702610914]
  This paper proposes the Flexible Automatic Identification and Removal (FAIR)-Pruner, a novel method for neural network structured pruning.<n>A major advantage of FAIR-Pruner lies in its capacity to automatically determine the layer-wise pruning rates, which yields a more efficient subnetwork structure.<n>With utilization scores and reconstruction errors, users can flexibly obtain pruned models under different pruning ratios.
  arXiv  Detail & Related papers  (2025-08-04T10:59:07Z)
• Loss-Aware Automatic Selection of Structured Pruning Criteria for Deep Neural Network Acceleration [1.3225694028747144]
  This paper presents an efficient Loss-Aware Automatic Selection of Structured Pruning Criteria (LAASP) for slimming and accelerating deep neural networks.<n>The pruning-while-training approach eliminates the first stage and integrates the second and third stages into a single cycle.<n>Experiments on the VGGNet and ResNet models on the CIFAR-10 and ImageNet benchmark datasets demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2025-06-25T06:18:46Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions. We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• Data-Free Dynamic Compression of CNNs for Tractable Efficiency [46.498278084317704]
  structured pruning approaches have shown promise in lowering floating-point operations without substantial drops in accuracy. We propose HASTE (Hashing for Tractable Efficiency), a data-free, plug-and-play convolution module that instantly reduces a network's test-time inference cost without training or fine-tuning. We demonstrate our approach on the popular vision benchmarks CIFAR-10 and ImageNet, where we achieve a 46.72% reduction in FLOPs with only a 1.25% loss in accuracy.
  arXiv  Detail & Related papers  (2023-09-29T13:09:40Z)
• Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning [12.982673904306633]
  We propose a novel framework named Unified Data-Free Compression(UDFC), which performs pruning and quantization simultaneously without any data and fine-tuning process. We evaluate the UDFC on the large-scale image classification task and obtain significant improvements over various network architectures and compression methods.
  arXiv  Detail & Related papers  (2023-08-14T15:25:07Z)
• Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer [60.31021888394358]
  Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image Super-Resolution (SR) We propose a SOurce-free Domain Adaptation framework for image SR (SODA-SR) to address this issue, i.e., adapt a source-trained model to a target domain with only unlabeled target data.
  arXiv  Detail & Related papers  (2023-03-31T03:14:44Z)
• Prompt Tuning for Parameter-efficient Medical Image Segmentation [79.09285179181225]
  We propose and investigate several contributions to achieve a parameter-efficient but effective adaptation for semantic segmentation on two medical imaging datasets. We pre-train this architecture with a dedicated dense self-supervision scheme based on assignments to online generated prototypes. We demonstrate that the resulting neural network model is able to attenuate the gap between fully fine-tuned and parameter-efficiently adapted models.
  arXiv  Detail & Related papers  (2022-11-16T21:55:05Z)
• Receptive Field-based Segmentation for Distributed CNN Inference Acceleration in Collaborative Edge Computing [93.67044879636093]
  We study inference acceleration using distributed convolutional neural networks (CNNs) in collaborative edge computing network. We propose a novel collaborative edge computing using fused-layer parallelization to partition a CNN model into multiple blocks of convolutional layers.
  arXiv  Detail & Related papers  (2022-07-22T18:38:11Z)
• Layer-Parallel Training of Residual Networks with Auxiliary-Variable Networks [28.775355111614484]
  auxiliary-variable methods have attracted much interest lately but suffer from significant communication overhead and lack of data augmentation. We present a novel joint learning framework for training realistic ResNets across multiple compute devices. We demonstrate the effectiveness of our methods on ResNets and WideResNets across CIFAR-10, CIFAR-100, and ImageNet datasets.
  arXiv  Detail & Related papers  (2021-12-10T08:45:35Z)
• Self-Supervised Pre-Training for Transformer-Based Person Re-Identification [54.55281692768765]
  Transformer-based supervised pre-training achieves great performance in person re-identification (ReID) Due to the domain gap between ImageNet and ReID datasets, it usually needs a larger pre-training dataset to boost the performance. This work aims to mitigate the gap between the pre-training and ReID datasets from the perspective of data and model structure.
  arXiv  Detail & Related papers  (2021-11-23T18:59:08Z)
• Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networks [0.9712140341805068]
  A highly efficient pruning method is proposed to significantly reduce the cost of pruning DCNN. Our method shows competitive pruning performance among the state-of-the-art retraining-based pruning methods.
  arXiv  Detail & Related papers  (2021-08-31T10:17:36Z)
• AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation [3.118384520557952]
  AutoAugment has sparked an interest in automated augmentation methods for deep learning models. We show that those methods are not robust when applied to biased and noisy data. We reformulate AutoAugment as a generalized automated dataset optimization (AutoDO) task. Our experiments show up to 9.3% improvement for biased datasets with label noise compared to prior methods.
  arXiv  Detail & Related papers  (2021-03-10T04:05:33Z)
• Manifold Regularized Dynamic Network Pruning [102.24146031250034]
  This paper proposes a new paradigm that dynamically removes redundant filters by embedding the manifold information of all instances into the space of pruned networks. The effectiveness of the proposed method is verified on several benchmarks, which shows better performance in terms of both accuracy and computational cost.
  arXiv  Detail & Related papers  (2021-03-10T03:59:03Z)
• DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search [55.164053971213576]
  convolutional neural network has achieved great success in fulfilling computer vision tasks despite large computation overhead. Structured (channel) pruning is usually applied to reduce the model redundancy while preserving the network structure. Existing structured pruning methods require hand-crafted rules which may lead to tremendous pruning space.
  arXiv  Detail & Related papers  (2020-11-04T07:43:01Z)
• Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge [11.789983276366987]
  This paper provides a first step towards enabling CNN finetuning on an edge device based on structured pruning. It explores the performance gains and costs of doing so and presents an open-source framework that allows the deployment of such approaches.
  arXiv  Detail & Related papers  (2020-06-15T17:16:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.