ShapeFlow: Dynamic Shape Interpreter for TensorFlow

• URL: http://arxiv.org/abs/2011.13452v1
• Date: Thu, 26 Nov 2020 19:27:25 GMT
• Title: ShapeFlow: Dynamic Shape Interpreter for TensorFlow
• Authors: Sahil Verma and Zhendong Su
• Abstract summary: We present ShapeFlow, a dynamic abstract interpreter for which quickly catches shape incompatibility errors. ShapeFlow constructs a custom shape computational graph, similar to the computational graph used by the programmer. We evaluate ShapeFlow on 52 programs collected by prior empirical studies to show how fast and accurately it can catch shape incompatibility errors.
• Score: 10.59840927423059
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present ShapeFlow, a dynamic abstract interpreter for TensorFlow which quickly catches tensor shape incompatibility errors, one of the most common bugs in deep learning code. ShapeFlow shares the same APIs as TensorFlow but only captures and emits tensor shapes, its abstract domain. ShapeFlow constructs a custom shape computational graph, similar to the computational graph used by TensorFlow. ShapeFlow requires no code annotation or code modification by the programmer, and therefore is convenient to use. We evaluate ShapeFlow on 52 programs collected by prior empirical studies to show how fast and accurately it can catch shape incompatibility errors compared to TensorFlow. We use two baselines: a worst-case training dataset size and a more realistic dataset size. ShapeFlow detects shape incompatibility errors highly accurately -- with no false positives and a single false negative -- and highly efficiently -- with an average speed-up of 499X and 24X for the first and second baseline, respectively. We believe ShapeFlow is a practical tool that benefits machine learning developers. We will open-source ShapeFlow on GitHub to make it publicly available to both the developer and research communities.

Related papers

• PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Accelerator [73.80050807279461]
  Piecewise Rectified Flow (PeRFlow) is a flow-based method for accelerating diffusion models. PeRFlow achieves superior performance in a few-step generation.
  arXiv  Detail & Related papers  (2024-05-13T07:10:53Z)
• PaddingFlow: Improving Normalizing Flows with Padding-Dimensional Noise [4.762593660623934]
  We propose PaddingFlow, a novel dequantization method, which improves normalizing flows with padding-dimensional noise. We validate our method on the main benchmarks of unconditional density estimation. The results show that PaddingFlow can perform better in all experiments in this paper.
  arXiv  Detail & Related papers  (2024-03-13T03:28:39Z)
• Expected flow networks in stochastic environments and two-player zero-sum games [63.98522423072093]
  Generative flow networks (GFlowNets) are sequential sampling models trained to match a given distribution. We propose expected flow networks (EFlowNets) which extend GFlowNets to environments. We show that EFlowNets outperform other GFlowNet formulations in tasks such as protein design. We then extend the concept of EFlowNets to adversarial environments, proposing adversarial flow networks (AFlowNets) for two-player zero-sum games.
  arXiv  Detail & Related papers  (2023-10-04T12:50:29Z)
• Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow [50.691232400959656]
  Trieste is an open-source Python package for Bayesian optimization and active learning. Our library enables the plug-and-play of popular models within sequential decision-making loops.
  arXiv  Detail & Related papers  (2023-02-16T17:21:49Z)
• OneFlow: Redesign the Distributed Deep Learning Framework from Scratch [17.798586916628174]
  OneFlow is a novel distributed training framework based on an SBP (split, broadcast and partial-value) abstraction and the actor model. SBP enables much easier programming of data parallelism and model parallelism than existing frameworks. OneFlow outperforms many well-known customized libraries built on top of the state-of-the-art frameworks.
  arXiv  Detail & Related papers  (2021-10-28T11:32:14Z)
• DeepLab2: A TensorFlow Library for Deep Labeling [118.95446843615049]
  DeepLab2 is a library for deep labeling for general dense pixel prediction problems in computer vision. DeepLab2 includes all our recently developed DeepLab model variants with pretrained checkpoints as well as model training and evaluation code. To showcase the effectiveness of DeepLab2, our Panoptic-DeepLab employing Axial-SWideRNet as network backbone achieves 68.0% PQ or 83.5% mIoU on Cityscaspes validation set.
  arXiv  Detail & Related papers  (2021-06-17T18:04:53Z)
• Implicit Normalizing Flows [43.939289514978434]
  ImpFlows generalize normalizing flows by allowing the mapping to be implicitly defined by the roots of an equation. We show that the function space of ImpFlow is strictly richer than that of ResFlows. We propose a scalable algorithm to train and draw samples from ImpFlows.
  arXiv  Detail & Related papers  (2021-03-17T09:24:04Z)
• DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows [145.83812019515818]
  We propose DeFlow, a method for learning image degradations from unpaired data. We model the degradation process in the latent space of a shared flow-decoder network. We validate our DeFlow formulation on the task of joint image restoration and super-resolution.
  arXiv  Detail & Related papers  (2021-01-14T18:58:01Z)
• OneFlow: One-class flow for anomaly detection based on a minimal volume region [12.691473293758607]
  OneFlow is a flow-based one-class classifier for anomaly (outlier) detection. It is constructed in such a way that its result does not depend on the structure of outliers. The proposed model outperforms related methods on real-world anomaly detection problems.
  arXiv  Detail & Related papers  (2020-10-06T20:09:11Z)
• VirtualFlow: Decoupling Deep Learning Models from the Underlying Hardware [9.461227523454188]
  State-of-the-art deep learning systems tightlycouple the model with the underlying hardware. We propose VirtualFlow to decouple the model from the hardware. In each step of training or inference, the batch of input data is split across virtual nodes instead of hardware accelerators.
  arXiv  Detail & Related papers  (2020-09-20T20:49:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.