TorchBench: Benchmarking PyTorch with High API Surface Coverage
- URL: http://arxiv.org/abs/2304.14226v3
- Date: Sat, 24 Jun 2023 16:57:43 GMT
- Title: TorchBench: Benchmarking PyTorch with High API Surface Coverage
- Authors: Yueming Hao, Xu Zhao, Bin Bao, David Berard, Will Constable, Adnan
Aziz, Xu Liu
- Abstract summary: We propose TorchBench, a novel benchmark suite to study the performance of PyTorch software stack.
TorchBench is able to comprehensively characterize the performance of the PyTorch software stack.
We show two practical use cases of TorchBench.
- Score: 9.68698340637426
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep learning (DL) has been a revolutionary technique in various domains. To
facilitate the model development and deployment, many deep learning frameworks
are proposed, among which PyTorch is one of the most popular solutions. The
performance of ecosystem around PyTorch is critically important, which saves
the costs of training models and reduces the response time of model inferences.
In this paper, we propose TorchBench, a novel benchmark suite to study the
performance of PyTorch software stack. Unlike existing benchmark suites,
TorchBench encloses many representative models, covering a large PyTorch API
surface. TorchBench is able to comprehensively characterize the performance of
the PyTorch software stack, guiding the performance optimization across models,
PyTorch framework, and GPU libraries. We show two practical use cases of
TorchBench. (1) We profile TorchBench to identify GPU performance
inefficiencies in PyTorch. We are able to optimize many performance bugs and
upstream patches to the official PyTorch repository. (2) We integrate
TorchBench into PyTorch continuous integration system. We are able to identify
performance regression in multiple daily code checkins to prevent PyTorch
repository from introducing performance bugs. TorchBench is open source and
keeps evolving.
Related papers
- Scorch: A Library for Sparse Deep Learning [41.62614683452247]
We introduce Scorch, a library that seamlessly integrates efficient sparse tensor computation into the PyTorch ecosystem.
Scorcher provides a flexible and intuitive interface for sparse tensors, supporting diverse sparse data structures.
We demonstrate Scorch's ease of use and performance gains on diverse deep learning models across multiple domains.
arXiv Detail & Related papers (2024-05-27T06:59:20Z) - depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers [92.13613958373628]
textttdepyf is a tool designed to demystify the inner workings of the PyTorch compiler.
textttdepyf decompiles bytecode generated by PyTorch back into equivalent source code.
arXiv Detail & Related papers (2024-03-14T16:17:14Z) - TorchCP: A Library for Conformal Prediction based on PyTorch [9.295285907724672]
TorchCP is a Python toolbox for conformal prediction research on deep learning models.
It contains various implementations for posthoc and training methods for classification and regression tasks.
arXiv Detail & Related papers (2024-02-20T03:14:47Z) - torchgfn: A PyTorch GFlowNet library [56.071033896777784]
torchgfn is a PyTorch library that aims to address this need.
It provides users with a simple API for environments and useful abstractions for samplers and losses.
arXiv Detail & Related papers (2023-05-24T00:20:59Z) - Torchhd: An Open Source Python Library to Support Research on
Hyperdimensional Computing and Vector Symbolic Architectures [99.70485761868193]
We present Torchhd, a high-performance open source Python library for HD/VSA.
Torchhd seeks to make HD/VSA more accessible and serves as an efficient foundation for further research and application development.
arXiv Detail & Related papers (2022-05-18T20:34:25Z) - PyGOD: A Python Library for Graph Outlier Detection [56.33769221859135]
PyGOD is an open-source library for detecting outliers in graph data.
It supports a wide array of leading graph-based methods for outlier detection.
PyGOD is released under a BSD 2-Clause license at https://pygod.org and at the Python Package Index (PyPI)
arXiv Detail & Related papers (2022-04-26T06:15:21Z) - PyHHMM: A Python Library for Heterogeneous Hidden Markov Models [63.01207205641885]
PyHHMM is an object-oriented Python implementation of Heterogeneous-Hidden Markov Models (HHMMs)
PyHHMM emphasizes features not supported in similar available frameworks: a heterogeneous observation model, missing data inference, different model order selection criterias, and semi-supervised training.
PyHHMM relies on the numpy, scipy, scikit-learn, and seaborn Python packages, and is distributed under the Apache-2.0 License.
arXiv Detail & Related papers (2022-01-12T07:32:36Z) - PyTorchVideo: A Deep Learning Library for Video Understanding [71.89124881732015]
PyTorchVideo is an open-source deep-learning library for video understanding tasks.
It covers a full stack of video understanding tools including multimodal data loading, transformations, and models.
The library is based on PyTorch and can be used by any training framework.
arXiv Detail & Related papers (2021-11-18T18:59:58Z) - Using Python for Model Inference in Deep Learning [0.6027358520885614]
We show how it is possible to meet performance and packaging constraints while performing inference in Python.
We present a way of using multiple Python interpreters within a single process to achieve scalable inference.
arXiv Detail & Related papers (2021-04-01T04:48:52Z) - TorchKGE: Knowledge Graph Embedding in Python and PyTorch [0.0]
TorchKGE is a Python module for knowledge graph (KG) embedding relying solely on PyTorch.
It features a KG data structure, simple model interfaces and modules for negative sampling and model evaluation.
arXiv Detail & Related papers (2020-09-07T09:21:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.