Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality

• URL: http://arxiv.org/abs/2407.05466v1
• Date: Sun, 7 Jul 2024 18:39:27 GMT
• Title: Studying the Impact of TensorFlow and PyTorch Bindings on Machine Learning Software Quality
• Authors: Hao Li, Gopi Krishnan Rajbahadur, Cor-Paul Bezemer,
• Abstract summary: We study the impact of using bindings in C#, Rust, Python and JavaScript on the software quality. Our experiments show that a model can be trained in one binding and used for inference in another binding for the same framework without losing accuracy.
• Score: 13.098132379075603
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bindings for machine learning frameworks (such as TensorFlow and PyTorch) allow developers to integrate a framework's functionality using a programming language different from the framework's default language (usually Python). In this paper, we study the impact of using TensorFlow and PyTorch bindings in C#, Rust, Python and JavaScript on the software quality in terms of correctness (training and test accuracy) and time cost (training and inference time) when training and performing inference on five widely used deep learning models. Our experiments show that a model can be trained in one binding and used for inference in another binding for the same framework without losing accuracy. Our study is the first to show that using a non-default binding can help improve machine learning software quality from the time cost perspective compared to the default Python binding while still achieving the same level of correctness.

Related papers

• A Comparative Survey of PyTorch vs TensorFlow for Deep Learning: Usability, Performance, and Deployment Trade-offs [0.0]
  We review each framework's programming paradigm and developer experience, contrasting PyTorch's graph-based approach with PyTorch's dynamic, Pythonic style.<n>We compare model training speeds and inference performance across multiple tasks and data regimes, drawing on recent benchmarks and studies.<n>Our findings indicate that while both frameworks are highly capable for state-of-the-art deep learning, they exhibit distinct trade-offs.
  arXiv  Detail & Related papers  (2025-08-06T02:55:57Z)
• PyPulse: A Python Library for Biosignal Imputation [58.35269251730328]
  We introduce PyPulse, a Python package for imputation of biosignals in both clinical and wearable sensor settings. PyPulse's framework provides a modular and extendable framework with high ease-of-use for a broad userbase, including non-machine-learning bioresearchers. We released PyPulse under the MIT License on Github and PyPI.
  arXiv  Detail & Related papers  (2024-12-09T11:00:55Z)
• Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning [50.26965628047682]
  Adapting pre-trained models to open classes is a challenging problem in machine learning. In this paper, we consider combining the advantages of both and come up with a test-time prompt tuning approach. Our proposed method outperforms all comparison methods on average considering both base and new classes.
  arXiv  Detail & Related papers  (2024-08-29T12:34:01Z)
• Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Images [76.47980643420375]
  This paper builds on the hypothesis that there is an inherent data-hungry matter in learning semantic correspondences. We demonstrate a simple machine annotator reliably enriches paired key points via machine supervision. Our models surpass current state-of-the-art models on semantic correspondence learning benchmarks like SPair-71k, PF-PASCAL, and PF-WILLOW.
  arXiv  Detail & Related papers  (2023-11-30T13:22:15Z)
• PyPOTS: A Python Toolkit for Machine Learning on Partially-Observed Time Series [20.491714178518155]
  PyPOTS is an open-source library for data mining and analysis.<n>It provides easy access to diverse algorithms categorized into five tasks.<n>PyPOTS is available on PyPI, Anaconda, and Docker.
  arXiv  Detail & Related papers  (2023-05-30T07:57:05Z)
• Transactional Python for Durable Machine Learning: Vision, Challenges, and Feasibility [5.669983975369642]
  Python applications may lose important data, such as trained models and extracted features, due to machine failures or human errors. This paper presents our vision of transactional Python that provides DART without any code modifications to user programs or the Python kernel. Our evaluation of a proof-of-concept implementation with public PyTorch and scikit-learn applications shows that DART can be offered with overheads ranging 1.5%--15.6%.
  arXiv  Detail & Related papers  (2023-05-15T16:27:09Z)
• T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics [94.69907794006826]
  We present a framework that combines the best of both worlds, using both supervised and unsupervised signals from whatever data we have available. We operationalize this idea by training T5Score, a metric that uses these training signals with mT5 as the backbone. T5Score achieves the best performance on all datasets against existing top-scoring metrics at the segment level.
  arXiv  Detail & Related papers  (2022-12-12T06:29:04Z)
• Latte: Cross-framework Python Package for Evaluation of Latent-Based Generative Models [65.51757376525798]
  Latte is a Python library for evaluation of latent-based generative models. Latte is compatible with both PyTorch and/Keras, and provides both functional and modular APIs.
  arXiv  Detail & Related papers  (2021-12-20T16:00:28Z)
• torch.fx: Practical Program Capture and Transformation for Deep Learning in Python [0.0]
  We study the different designs for program capture and transformation used in deep learning. By designing for typical deep learning use cases rather than long tail ones, it is possible to create a simpler framework for program capture and transformation. We apply this principle in torch.fx, a program capture and transformation library for PyTorch written entirely in Python and optimized for high developer productivity by ML practitioners.
  arXiv  Detail & Related papers  (2021-12-15T19:16:29Z)
• COMBO: State-of-the-Art Morphosyntactic Analysis [0.0]
  COMBO is a fully neural NLP system for accurate part-of-speech tagging, morphological analysis, lemmatisation, and (enhanced) dependency parsing. It predicts categorical morphosyntactic features whilst also exposing their vector representations, extracted from hidden layers. It is an easy to install Python package with automatically downloadable pre-trained models for over 40 languages.
  arXiv  Detail & Related papers  (2021-09-11T20:00:20Z)
• Using Python for Model Inference in Deep Learning [0.6027358520885614]
  We show how it is possible to meet performance and packaging constraints while performing inference in Python. We present a way of using multiple Python interpreters within a single process to achieve scalable inference.
  arXiv  Detail & Related papers  (2021-04-01T04:48:52Z)
• Do Adversarially Robust ImageNet Models Transfer Better? [102.09335596483695]
  adversarially robust models often perform better than their standard-trained counterparts when used for transfer learning. Our results are consistent with (and in fact, add to) recent hypotheses stating that robustness leads to improved feature representations.
  arXiv  Detail & Related papers  (2020-07-16T17:42:40Z)
• The Right Tool for the Job: Matching Model and Instance Complexities [62.95183777679024]
  As NLP models become larger, executing a trained model requires significant computational resources incurring monetary and environmental costs. We propose a modification to contextual representation fine-tuning which, during inference, allows for an early (and fast) "exit" We test our proposed modification on five different datasets in two tasks: three text classification datasets and two natural language inference benchmarks.
  arXiv  Detail & Related papers  (2020-04-16T04:28:08Z)
• TF-Coder: Program Synthesis for Tensor Manipulations [29.46838583290554]
  We present a tool called TF-Coder for programming by example in pruning. We train models to predict operations from features of the input and output tensors and natural language descriptions of tasks. TF-Coder solves 63 of 70 real-world tasks within 5 minutes, sometimes finding simpler solutions in less time compared to experienced human programmers.
  arXiv  Detail & Related papers  (2020-03-19T22:53:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.