CWD: A Machine Learning based Approach to Detect Unknown Cloud Workloads

• URL: http://arxiv.org/abs/2211.15739v1
• Date: Mon, 28 Nov 2022 19:41:56 GMT
• Title: CWD: A Machine Learning based Approach to Detect Unknown Cloud Workloads
• Authors: Mohammad Hossain, Derssie Mebratu, Niranjan Hasabnis, Jun Jin, Gaurav Chaudhary, Noah Shen
• Abstract summary: We develop a machine learning based technique to characterize, profile and predict workloads running in the cloud environment. We also develop techniques to analyze the performance of the model in a standalone manner.
• Score: 3.523208537466129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Workloads in modern cloud data centers are becoming increasingly complex. The number of workloads running in cloud data centers has been growing exponentially for the last few years, and cloud service providers (CSP) have been supporting on-demand services in real-time. Realizing the growing complexity of cloud environment and cloud workloads, hardware vendors such as Intel and AMD are increasingly introducing cloud-specific workload acceleration features in their CPU platforms. These features are typically targeted towards popular and commonly-used cloud workloads. Nonetheless, uncommon, customer-specific workloads (unknown workloads), if their characteristics are different from common workloads (known workloads), may not realize the potential of the underlying platform. To address this problem of realizing the full potential of the underlying platform, we develop a machine learning based technique to characterize, profile and predict workloads running in the cloud environment. Experimental evaluation of our technique demonstrates good prediction performance. We also develop techniques to analyze the performance of the model in a standalone manner.

Related papers

• SeBS-Flow: Benchmarking Serverless Cloud Function Workflows [51.4200085836966]
  We propose the first serverless workflow benchmarking suite SeBS-Flow. SeBS-Flow includes six real-world application benchmarks and four microbenchmarks representing different computational patterns. We conduct comprehensive evaluations on three major cloud platforms, assessing performance, cost, scalability, and runtime deviations.
  arXiv  Detail & Related papers  (2024-10-04T14:52:18Z)
• Teaching Cloud Infrastructure and Scalable Application Deployment in an Undergraduate Computer Science Program [2.8912542516745168]
  Students commonly interact with cloud abstractions early in their technical careers. Building cloud-native applications without a firm understanding of the fundamentals of cloud engineering can leave students susceptible to cost and security pitfalls common to cloud platforms. Our course featured a number of hands-on assignments that gave students experience with modern, best-practice concepts and tools such as IaC, containerization, observability, serverless computing, and continuous integration and deployment.
  arXiv  Detail & Related papers  (2024-10-01T19:49:19Z)
• Integrating Homomorphic Encryption and Trusted Execution Technology for Autonomous and Confidential Model Refining in Cloud [4.21388107490327]
  Homomorphic encryption and trusted execution environment technology can protect confidentiality for autonomous computation. We propose to integrate these two techniques in the design of the model refining scheme.
  arXiv  Detail & Related papers  (2023-08-02T06:31:41Z)
• Managing Cold-start in The Serverless Cloud with Temporal Convolutional Networks [0.0]
  Serverless cloud is an innovative cloud service model that frees customers from most cloud management duties. A big threat to the serverless cloud's performance is cold-start, which is when the time of provisioning the needed cloud resource to serve customers' requests incurs unacceptable costs to the service providers and/or the customers. This paper proposes a novel low-coupling, high-cohesion ensemble policy that addresses the cold-start problem at infrastructure- and function-levels of the serverless cloud stack.
  arXiv  Detail & Related papers  (2023-04-01T21:54:22Z)
• Measuring the Carbon Intensity of AI in Cloud Instances [91.28501520271972]
  We provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions. We evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform.
  arXiv  Detail & Related papers  (2022-06-10T17:04:04Z)
• Molecular Dynamics Simulations on Cloud Computing and Machine Learning Platforms [0.8093262393618671]
  We see a paradigm shift in the computational structure, design, and requirements of scientific computing applications. Data-driven and machine learning approaches are being used to support, speed-up, and enhance scientific computing applications. Cloud computing platforms are increasingly appealing for scientific computing.
  arXiv  Detail & Related papers  (2021-11-11T21:20:26Z)
• Edge-Cloud Polarization and Collaboration: A Comprehensive Survey [61.05059817550049]
  We conduct a systematic review for both cloud and edge AI. We are the first to set up the collaborative learning mechanism for cloud and edge modeling. We discuss potentials and practical experiences of some on-going advanced edge AI topics.
  arXiv  Detail & Related papers  (2021-11-11T05:58:23Z)
• Auto-Split: A General Framework of Collaborative Edge-Cloud AI [49.750972428032355]
  This paper describes the techniques and engineering practice behind Auto-Split, an edge-cloud collaborative prototype of Huawei Cloud. To the best of our knowledge, there is no existing industry product that provides the capability of Deep Neural Network (DNN) splitting.
  arXiv  Detail & Related papers  (2021-08-30T08:03:29Z)
• Device-Cloud Collaborative Learning for Recommendation [50.01289274123047]
  We propose a novel MetaPatch learning approach on the device side to efficiently achieve "thousands of people with thousands of models" given a centralized cloud model. With billions of updated personalized device models, we propose a "model-over-models" distillation algorithm, namely MoMoDistill, to update the centralized cloud model.
  arXiv  Detail & Related papers  (2021-04-14T05:06:59Z)
• Machine Learning Algorithms for Active Monitoring of High Performance Computing as a Service (HPCaaS) Cloud Environments [0.0]
  This paper explores the viability of engineering applications running on a cloud infrastructure configured as an HPC platform. The engineering applications considered in this work include MCNP6, a radiation transport code developed by Los Alamos National Laboratory, OpenFOAM, an open source computational fluid dynamics code, and CADONFS, a numerical implementation of the general number field sieve algorithm used for prime number factorization.
  arXiv  Detail & Related papers  (2020-09-26T01:29:19Z)
• A Privacy-Preserving Distributed Architecture for Deep-Learning-as-a-Service [68.84245063902908]
  This paper introduces a novel distributed architecture for deep-learning-as-a-service. It is able to preserve the user sensitive data while providing Cloud-based machine and deep learning services.
  arXiv  Detail & Related papers  (2020-03-30T15:12:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.