APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning

• URL: http://arxiv.org/abs/2608.05499v1
• Date: Thu, 06 Aug 2026 01:09:50 GMT
• Title: APQF: Agentic Profiling-Guided Structured Pruning and Mixed-Precision Quantization with Adaptive Fine-Tuning
• Abstract summary: APQF is a profiling-guided framework for deep neural networks.<n>It combines structured pruning, mixed-precision quantization-aware training, and accuracy recovery in one automated pipeline.<n>We evaluate it on ResNet, VGG7, ViT, DeiT, and Swin using ImageNet-1k and CIFAR-10.
• Score: 12.834835889446177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern deep neural networks achieve strong performance, but their scale makes them costly and slow, especially on resource-constrained edge devices. Pruning and quantization address this, but rely on manual, expert choices and on algorithms that are hard to apply across architectures. Uniform settings also ignore how differently individual layers respond to compression, which costs accuracy. We introduce APQF, an agentic profiling-guided framework that combines structured pruning, mixed-precision quantization-aware training, and accuracy recovery in one automated pipeline. A profiling agent measures how cost is distributed across the model and how sensitive each part is to pruning, and this evidence drives per-layer pruning ratios, per-layer bit-widths, and the recovery strategy, all proposed by LLM planners and validated before execution. To our knowledge, APQF is the first framework to combine LLM-guided, profiling-grounded decisions with a fully training-aware pruning and quantization pipeline for both CNNs and vision transformers. We evaluate APQF on ResNet, VGG7, ViT, DeiT, and Swin using ImageNet-1k and CIFAR-10. On ImageNet it cuts compute to 5.6-7.7 percent of the original bit-operations, a 13-18x reduction, while keeping accuracy close to the baseline, and under a 200K-image budget it stays roughly 17 points higher in Top-1 than existing joint pruning and quantization methods. On CIFAR-10 it compresses further than that method on four of five architectures. On VGG7 it reaches 93.15 percent using only 0.41 percent of baseline bit-operations, the only method at that compression level to improve on its full-precision baseline. Ablations show that uniform compression loses the most accuracy at matched compute, and that withholding profiling data from the planner hurts every model. Six LLM planners, including free open-weight ones, all reach 97.4-97.9 percent on Swin-Tiny.

Related papers

• RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference [1.1100764382749708]
  RAMP (Reinforcement Adaptive Mixed Precision) learns per layer bit width assignments to minimize perplexity under a global bit budget.<n>On Llama 2 7B, RAMP achieves 5.54 perplexity at 3.68GB (3.65 effective bits), outperforming uniform 4 bit AWQ (5.60 at 3.90 GB) and GPTQ by 6% in size and 1% to3% in quality.
  arXiv  Detail & Related papers  (2026-03-18T16:16:28Z)
• ProfilingAgent: Profiling-Guided Agentic Reasoning for Adaptive Model Optimization [7.64805011214817]
  Profiling tools expose per-layer latency, memory, and compute cost, yet are rarely integrated into automated pipelines.<n>We propose ProfilingAgent, a profiling-guided, agentic approach that uses large language models (LLMs) to automate compression via structured pruning and post-training dynamic quantization.
  arXiv  Detail & Related papers  (2025-09-06T04:02:04Z)
• Joint Pruning and Channel-wise Mixed-Precision Quantization for Efficient Deep Neural Networks [10.229120811024162]
  deep neural networks (DNNs) pose significant challenges to their deployment on edge devices. Common approaches to address this issue are pruning and mixed-precision quantization. We propose a novel methodology to apply them jointly via a lightweight gradient-based search.
  arXiv  Detail & Related papers  (2024-07-01T08:07:02Z)
• Filter Pruning For CNN With Enhanced Linear Representation Redundancy [3.853146967741941]
  We present a data-driven loss function term calculated from the correlation matrix of different feature maps in the same layer, named CCM-loss. CCM-loss provides us with another universal transcendental mathematical tool besides L*-norm regularization. In our new strategy, we mainly focus on the consistency and integrality of the information flow in the network.
  arXiv  Detail & Related papers  (2023-10-10T06:27:30Z)
• LAPP: Layer Adaptive Progressive Pruning for Compressing CNNs from Scratch [14.911305800463285]
  We propose a novel framework named Layer Adaptive Progressive Pruning (LAPP) LAPP designs an effective and efficient pruning strategy that introduces a learnable threshold for each layer and FLOPs constraints for network. Our method demonstrates superior performance gains over previous compression methods on various datasets and backbone architectures.
  arXiv  Detail & Related papers  (2023-09-25T14:08:45Z)
• Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networks [48.089501687522954]
  We propose a novel layer-adaptive weight-pruning approach for Deep Neural Networks (DNNs) Our approach takes into account the collective influence of all layers to design a layer-adaptive pruning scheme. Our experiments demonstrate the superiority of our approach over existing methods on the ImageNet and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2023-08-21T03:22:47Z)
• Pruning by Active Attention Manipulation [49.61707925611295]
  Filter pruning of a CNN is typically achieved by applying discrete masks on the CNN's filter weights or activation maps, post-training. Here, we present a new filter-importance-scoring concept named pruning by active attention manipulation (PAAM) PAAM learns analog filter scores from the filter weights by optimizing a cost function regularized by an additive term in the scores.
  arXiv  Detail & Related papers  (2022-10-20T09:17:02Z)
• End-to-End Sensitivity-Based Filter Pruning [49.61707925611295]
  We present a sensitivity-based filter pruning algorithm (SbF-Pruner) to learn the importance scores of filters of each layer end-to-end. Our method learns the scores from the filter weights, enabling it to account for the correlations between the filters of each layer.
  arXiv  Detail & Related papers  (2022-04-15T10:21:05Z)
• Sharpness-aware Quantization for Deep Neural Networks [45.150346855368]
  Sharpness-Aware Quantization (SAQ) is a novel method to explore the effect of Sharpness-Aware Minimization (SAM) on model compression. We show that SAQ improves the generalization performance of the quantized models, yielding the SOTA results in uniform quantization.
  arXiv  Detail & Related papers  (2021-11-24T05:16:41Z)
• Towards Compact CNNs via Collaborative Compression [166.86915086497433]
  We propose a Collaborative Compression scheme, which joints channel pruning and tensor decomposition to compress CNN models. We achieve 52.9% FLOPs reduction by removing 48.4% parameters on ResNet-50 with only a Top-1 accuracy drop of 0.56% on ImageNet 2012.
  arXiv  Detail & Related papers  (2021-05-24T12:07:38Z)
• Non-Parametric Adaptive Network Pruning [125.4414216272874]
  We introduce non-parametric modeling to simplify the algorithm design. Inspired by the face recognition community, we use a message passing algorithm to obtain an adaptive number of exemplars. EPruner breaks the dependency on the training data in determining the "important" filters.
  arXiv  Detail & Related papers  (2021-01-20T06:18:38Z)
• Training with Quantization Noise for Extreme Model Compression [57.51832088938618]
  We tackle the problem of producing compact models, maximizing their accuracy for a given model size. A standard solution is to train networks with Quantization Aware Training, where the weights are quantized during training and the gradients approximated with the Straight-Through Estimator. In this paper, we extend this approach to work beyond int8 fixed-point quantization with extreme compression methods.
  arXiv  Detail & Related papers  (2020-04-15T20:10:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.