Beyond One-Way Pruning: Bidirectional Pruning-Regrowth for Extreme Accuracy-Sparsity Tradeoff

• URL: http://arxiv.org/abs/2511.11675v1
• Date: Tue, 11 Nov 2025 21:22:29 GMT
• Title: Beyond One-Way Pruning: Bidirectional Pruning-Regrowth for Extreme Accuracy-Sparsity Tradeoff
• Abstract summary: When sparsity exceeds a certain threshold, both iterative and one-shot pruning methods lead to a steep decline in model performance.<n>This rapid degradation limits the achievable compression ratio and prevents models from meeting the stringent size constraints required by certain hardware platforms.<n>We propose a bidirectional pruning-regrowth strategy, starting from an extremely compressed network that satisfies hardware constraints.
• Score: 7.1603511160472
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a widely adopted model compression technique, model pruning has demonstrated strong effectiveness across various architectures. However, we observe that when sparsity exceeds a certain threshold, both iterative and one-shot pruning methods lead to a steep decline in model performance. This rapid degradation limits the achievable compression ratio and prevents models from meeting the stringent size constraints required by certain hardware platforms, rendering them inoperable. To overcome this limitation, we propose a bidirectional pruning-regrowth strategy. Starting from an extremely compressed network that satisfies hardware constraints, the method selectively regenerates critical connections to recover lost performance, effectively mitigating the sharp accuracy drop commonly observed under high sparsity conditions.

Related papers

• Breaking the Compression Barrier: Cross-Architecture Compression Boundary Learning via Reverse Regrowth [1.454787329315493]
  We propose a boundary-Learning reverse framework, BRIDGE, to reformulate compression as a constructive boundary-search problem.<n>Our method first drives the model to an extremely sparse state to expose the collapse region, and then selectively regenerates the critical structure to restore performance.<n>Experiments show that our method can recover models from the brink of collapse on both CNNs and Transformer architectures.
  arXiv  Detail & Related papers  (2026-08-17T01:59:23Z)
• PACE: Post-Causal Entropy Modeling for Learned LiDAR Point Cloud Compression [43.524231862354405]
  LiDAR point cloud compression is vital for autonomous systems to handle massive data from high-resolution sensors.<n>We propose PACE, a new framework that reformulates ancestral context aggregation as a non-causal backbone and causality to a lightweight, stage-scalable predictor.<n>Experiments demonstrate that PACE sets a new state-of-the-art in compression efficiency, achieving notable BD-BR savings and reducing decoding latency by over 90% in autoregressive mode.
  arXiv  Detail & Related papers  (2026-05-02T08:36:11Z)
• A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exits [0.08209843760716957]
  This paper presents a unified deployment-oriented comparison of static compression and dynamic early-exit mechanisms.<n>Our results show that static and dynamic techniques offer fundamentally different trade-offs for edge deployment.<n>Their combination proves highly effective, simultaneously reducing inference latency and memory usage with minimal accuracy loss.
  arXiv  Detail & Related papers  (2026-04-16T08:50:49Z)
• Gradually Compacting Large Language Models for Reasoning Like a Boiling Frog [72.4168434368873]
  Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, but their substantial size often demands significant computational resources.<n>We propose a gradual compacting method that divides the compression process into multiple fine-grained iterations.<n>This iterative approach-reminiscent of the "boiling frog" effect-enables the model to be progressively compressed without abrupt performance loss.
  arXiv  Detail & Related papers  (2026-02-04T06:56:52Z)
• Integrating Pruning with Quantization for Efficient Deep Neural Networks Compression [0.0]
  pruning and quantization are widely used compression techniques to reduce model size and enhance processing speed.<n>We propose two approaches that integrate similarity-based filter pruning with Adaptive Power-of-Two (APoT) quantization to achieve higher compression efficiency.<n> Experimental results demonstrate that our proposed approaches achieve effective model compression with minimal accuracy degradation.
  arXiv  Detail & Related papers  (2025-09-04T14:17:28Z)
• Towards Universal & Efficient Model Compression via Exponential Torque Pruning [2.597821418338574]
  We propose Exponential Torque Pruning (ETP), which adopts an exponential force application scheme for regularization.<n>ETP manages to achieve significantly higher compression rate than the previous state-of-the-art pruning strategies with negligible accuracy drop.
  arXiv  Detail & Related papers  (2025-06-27T08:28:21Z)
• One-Step Diffusion Model for Image Motion-Deblurring [85.76149042561507]
  We propose a one-step diffusion model for deblurring (OSDD), a novel framework that reduces the denoising process to a single step.<n>To tackle fidelity loss in diffusion models, we introduce an enhanced variational autoencoder (eVAE), which improves structural restoration.<n>Our method achieves strong performance on both full and no-reference metrics.
  arXiv  Detail & Related papers  (2025-03-09T09:39:57Z)
• Choose Your Model Size: Any Compression of Large Language Models Without Re-Computation [10.376875638696504]
  This work presents Any Compression via Iterative Pruning (ACIP), a novel algorithmic approach to determine a compression-performance trade-off.<n>We use an SVD-reparametrization of linear layers and iteratively prune their singular values with a sparsity-inducing penalty.<n>We show that ACIP seamlessly complements common quantization-based compression techniques.
  arXiv  Detail & Related papers  (2025-02-03T18:40:58Z)
• Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models [93.76814568163353]
  We propose a novel bilevel optimization framework for pruned diffusion models.<n>This framework consolidates the fine-tuning and unlearning processes into a unified phase.<n>It is compatible with various pruning and concept unlearning methods.
  arXiv  Detail & Related papers  (2024-12-19T19:13:18Z)
• Convex Distillation: Efficient Compression of Deep Networks via Convex Optimization [46.18363767705346]
  Deployment of large and complex convex networks on resource-constrained devices significant challenges due to their demands. We introduce novel distillation technique that efficiently compresses model via this model via this paper. Our approach enables performance comparable to original model without requiring any post-processing.
  arXiv  Detail & Related papers  (2024-10-09T06:04:52Z)
• Learning Accurate Performance Predictors for Ultrafast Automated Model Compression [86.22294249097203]
  We propose an ultrafast automated model compression framework called SeerNet for flexible network deployment. Our method achieves competitive accuracy-complexity trade-offs with significant reduction of the search cost.
  arXiv  Detail & Related papers  (2023-04-13T10:52:49Z)
• Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints [81.46143788046892]
  We focus on the task of controlling the level of sparsity when performing sparse learning. Existing methods based on sparsity-inducing penalties involve expensive trial-and-error tuning of the penalty factor. We propose a constrained formulation where sparsification is guided by the training objective and the desired sparsity target in an end-to-end fashion.
  arXiv  Detail & Related papers  (2022-08-08T21:24:20Z)
• Attentive Fine-Grained Structured Sparsity for Image Restoration [63.35887911506264]
  N:M structured pruning has appeared as one of the effective and practical pruning approaches for making the model efficient with the accuracy constraint. We propose a novel pruning method that determines the pruning ratio for N:M structured sparsity at each layer.
  arXiv  Detail & Related papers  (2022-04-26T12:44:55Z)
• Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm [7.662952656290564]
  Various pruning approaches have been proposed to reduce the footprint requirements of Transformer-based language models. We show for the first time that reducing the risk of overfitting can help the effectiveness of pruning under the pretrain-and-finetune paradigm.
  arXiv  Detail & Related papers  (2021-10-15T16:42:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.