PruningBench: A Comprehensive Benchmark of Structural Pruning

• URL: http://arxiv.org/abs/2406.12315v3
• Date: Sat, 20 Jul 2024 10:56:31 GMT
• Title: PruningBench: A Comprehensive Benchmark of Structural Pruning
• Authors: Haoling Li, Changhao Li, Mengqi Xue, Gongfan Fang, Sheng Zhou, Zunlei Feng, Huiqiong Wang, Yong Wang, Lechao Cheng, Mingli Song, Jie Song,
• Abstract summary: We present the first comprehensive benchmark, termed textitPruningBench, for structural pruning. PruningBench employs a unified and consistent framework for evaluating the effectiveness of diverse structural pruning techniques. It provides easily implementable interfaces to facilitate the implementation of future pruning methods, and enables the subsequent researchers to incorporate their work into our leaderboards.
• Score: 50.23493036025595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Structural pruning has emerged as a promising approach for producing more efficient models. Nevertheless, the community suffers from a lack of standardized benchmarks and metrics, leaving the progress in this area not fully comprehended. To fill this gap, we present the first comprehensive benchmark, termed \textit{PruningBench}, for structural pruning. PruningBench showcases the following three characteristics: 1) PruningBench employs a unified and consistent framework for evaluating the effectiveness of diverse structural pruning techniques; 2) PruningBench systematically evaluates 16 existing pruning methods, encompassing a wide array of models (e.g., CNNs and ViTs) and tasks (e.g., classification and detection); 3) PruningBench provides easily implementable interfaces to facilitate the implementation of future pruning methods, and enables the subsequent researchers to incorporate their work into our leaderboards. We provide an online pruning platform http://pruning.vipazoo.cn for customizing pruning tasks and reproducing all results in this paper. Codes will be made publicly on https://github.com/HollyLee2000/PruningBench.

Related papers

• ShuffleBench: A Benchmark for Large-Scale Data Shuffling Operations with Distributed Stream Processing Frameworks [1.4374467687356276]
  This paper introduces ShuffleBench, a novel benchmark to evaluate the performance of modern stream processing frameworks. ShuffleBench is inspired by requirements for near real-time analytics of a large cloud observability platform. Our results show that Flink achieves the highest throughput while Hazelcast processes data streams with the lowest latency.
  arXiv  Detail & Related papers  (2024-03-07T15:06:24Z)
• RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems [43.797002322559834]
  RepoBench is a benchmark for evaluating code auto-completion systems. It consists of three evaluation tasks: RepoBench-R (Retrieval), RepoBench-C (Code Completion), and RepoBench-P (Pipeline)
  arXiv  Detail & Related papers  (2023-06-05T17:59:41Z)
• Advancing Model Pruning via Bi-level Optimization [89.88761425199598]
  iterative magnitude pruning (IMP) is the predominant pruning method to successfully find 'winning tickets' One-shot pruning methods have been developed, but these schemes are usually unable to find winning tickets as good as IMP. We show that the proposed bi-level optimization-oriented pruning method (termed BiP) is a special class of BLO problems with a bi-linear problem structure.
  arXiv  Detail & Related papers  (2022-10-08T19:19:29Z)
• A Fast Post-Training Pruning Framework for Transformers [74.59556951906468]
  Pruning is an effective way to reduce the huge inference cost of large Transformer models. Prior work on model pruning requires retraining the model. We propose a fast post-training pruning framework for Transformers that does not require any retraining.
  arXiv  Detail & Related papers  (2022-03-29T07:41:11Z)
• Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking for Everyone [45.673814384050004]
  We introduce Codabench, an open-sourced, community-driven platform for benchmarking algorithms or software agents versus datasets or tasks. A public instance of Codabench is open to everyone, free of charge, and allows benchmark organizers to compare fairly submissions.
  arXiv  Detail & Related papers  (2021-10-12T07:54:34Z)
• MLPruning: A Multilevel Structured Pruning Framework for Transformer-based Models [78.45898846056303]
  Pruning is an effective method to reduce the memory footprint and computational cost associated with large natural language processing models. We develop a novel MultiLevel structured Pruning framework, which uses three different levels of structured pruning: head pruning, row pruning, and block-wise sparse pruning.
  arXiv  Detail & Related papers  (2021-05-30T22:00:44Z)
• Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot [55.37967301483917]
  Conventional wisdom of pruning algorithms suggests that pruning methods exploit information from training data to find goodworks. In this paper, we conduct sanity checks for the above beliefs on several recent unstructured pruning methods. We propose a series of simple emphdata-independent prune ratios for each layer, and randomly prune each layer accordingly to get a subnetwork.
  arXiv  Detail & Related papers  (2020-09-22T17:36:17Z)
• Image Matching across Wide Baselines: From Paper to Practice [80.9424750998559]
  We introduce a comprehensive benchmark for local features and robust estimation algorithms. Our pipeline's modular structure allows easy integration, configuration, and combination of different methods. We show that with proper settings, classical solutions may still outperform the perceived state of the art.
  arXiv  Detail & Related papers  (2020-03-03T15:20:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.