Deep Learning Model Deployment in Multiple Cloud Providers: an Exploratory Study Using Low Computing Power Environments

• URL: http://arxiv.org/abs/2503.23988v1
• Date: Mon, 31 Mar 2025 11:58:37 GMT
• Title: Deep Learning Model Deployment in Multiple Cloud Providers: an Exploratory Study Using Low Computing Power Environments
• Authors: Elayne Lemos, Rodrigo Oliveira, Jairson Rodrigues, Rosalvo F. Oliveira Neto,
• Abstract summary: This study demonstrates the feasibility and affordability of cloud-based Machine Learning inference solutions without GPU.<n>We evaluate real-time latency, hardware usage and cost at each cloud provider by 7 execution environments with 10 experiments reproduced.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The deployment of Machine Learning models at cloud have grown by tech companies. Hardware requirements are higher when these models involve Deep Learning (DL) techniques and the cloud providers' costs may be a barrier. We explore deploying DL models using for experiments the GECToR model, a DL solution for Grammatical Error Correction, across three of the major cloud platforms (AWS, Google Cloud, Azure). We evaluate real-time latency, hardware usage and cost at each cloud provider by 7 execution environments with 10 experiments reproduced. We found that while GPUs excel in performance, they had an average cost 300% higher than solutions without GPU. Our analysis also identifies that processor cache size is crucial for cost-effective CPU deployments, enabling over 50% of cost reduction compared to GPUs. This study demonstrates the feasibility and affordability of cloud-based DL inference solutions without GPUs, benefiting resource-constrained users like startups.

Related papers

• FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUs [57.12856172329322]
  We envision a decentralized system unlocking the potential vast untapped consumer-level GPU. This system faces critical challenges, including limited CPU and GPU memory, low network bandwidth, the variability of peer and device heterogeneity.
  arXiv  Detail & Related papers  (2023-09-03T13:27:56Z)
• How Can We Train Deep Learning Models Across Clouds and Continents? An Experimental Study [57.97785297481162]
  We evaluate the cost and throughput implications of training in different zones, continents, and clouds for representative CV, NLP, and ASR models. We show how leveraging spot pricing enables a new cost-efficient way to train models with multiple cheap instance, trumping both more centralized and powerful hardware and even on-demand cloud offerings at competitive prices.
  arXiv  Detail & Related papers  (2023-06-05T18:17:37Z)
• CWD: A Machine Learning based Approach to Detect Unknown Cloud Workloads [3.523208537466129]
  We develop a machine learning based technique to characterize, profile and predict workloads running in the cloud environment. We also develop techniques to analyze the performance of the model in a standalone manner.
  arXiv  Detail & Related papers  (2022-11-28T19:41:56Z)
• Analysis of Distributed Deep Learning in the Cloud [17.91202259637393]
  We introduce a comprehensive distributed deep learning (DDL) profiler, which can determine the various execution "stalls" that DDL suffers from while running on a public cloud. We estimate two types of communication stalls - interconnect and network stalls. We train popular DNN models using the profiler to characterize various AWS GPU instances and list their advantages and shortcomings for users to make an informed decision.
  arXiv  Detail & Related papers  (2022-08-30T15:42:36Z)
• Kubric: A scalable dataset generator [73.78485189435729]
  Kubric is a Python framework that interfaces with PyBullet and Blender to generate photo-realistic scenes, with rich annotations, and seamlessly scales to large jobs distributed over thousands of machines. We demonstrate the effectiveness of Kubric by presenting a series of 13 different generated datasets for tasks ranging from studying 3D NeRF models to optical flow estimation.
  arXiv  Detail & Related papers  (2022-03-07T18:13:59Z)
• ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning [141.58588761593955]
  We present a library ElegantRL-podracer for cloud-native deep reinforcement learning. It efficiently supports millions of cores to carry out massively parallel training at multiple levels. At a low-level, each pod simulates agent-environment interactions in parallel by fully utilizing nearly 7,000 GPU cores in a single GPU.
  arXiv  Detail & Related papers  (2021-12-11T06:31:21Z)
• Auto-Split: A General Framework of Collaborative Edge-Cloud AI [49.750972428032355]
  This paper describes the techniques and engineering practice behind Auto-Split, an edge-cloud collaborative prototype of Huawei Cloud. To the best of our knowledge, there is no existing industry product that provides the capability of Deep Neural Network (DNN) splitting.
  arXiv  Detail & Related papers  (2021-08-30T08:03:29Z)
• Sampling Training Data for Continual Learning Between Robots and the Cloud [26.116999231118793]
  We introduce HarvestNet, an intelligent sampling algorithm that resides on-board a robot and reduces system bottlenecks. It significantly improves the accuracy of machine-learning models on our novel dataset of road construction sites, field testing of self-driving cars, and streaming face recognition. It is between 1.05-2.58x more accurate than baseline algorithms and scalably runs on embedded deep learning hardware.
  arXiv  Detail & Related papers  (2020-12-12T05:52:33Z)
• Budget Learning via Bracketing [50.085728094234476]
  The budget learning problem poses the learner's goal as minimising use of the cloud while suffering no discernible loss in accuracy. We propose a new formulation for the BL problem via the concept of bracketings. We empirically validate our theory on real-world datasets, demonstrating improved performance over prior gating based methods.
  arXiv  Detail & Related papers  (2020-04-14T04:38:14Z)
• Characterizing and Modeling Distributed Training with Transient Cloud GPU Servers [6.56704851092678]
  We analyze distributed training performance under diverse cluster configurations using CM-DARE. Our empirical datasets include measurements from three GPU types, six geographic regions, twenty convolutional neural networks, and thousands of Google Cloud servers. We also demonstrate the feasibility of predicting training speed and overhead using regression-based models.
  arXiv  Detail & Related papers  (2020-04-07T01:49:58Z)
• ContainerStress: Autonomous Cloud-Node Scoping Framework for Big-Data ML Use Cases [0.2752817022620644]
  OracleLabs has developed an automated framework that uses nested-loop Monte Carlo simulation to autonomously scale any size customer ML use cases. OracleLabs and NVIDIA authors have collaborated on a ML benchmark study which analyzes the compute cost and GPU acceleration of any ML prognostic algorithm.
  arXiv  Detail & Related papers  (2020-03-18T01:51:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.