Large-Scale Intelligent Microservices

• URL: http://arxiv.org/abs/2009.08044v3
• Date: Thu, 2 Dec 2021 20:09:30 GMT
• Title: Large-Scale Intelligent Microservices
• Authors: Mark Hamilton, Nick Gonsalves, Christina Lee, Anand Raman, Brendan Walsh, Siddhartha Prasad, Dalitso Banda, Lucy Zhang, Mei Gao, Lei Zhang, William T. Freeman
• Abstract summary: We introduce an Apache Spark-based micro-service orchestration framework that extends database operations to include web service primitives. We provide large scale clients for intelligent services such as speech, vision, search, anomaly detection, and text analysis.
• Score: 24.99695289157708
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying Machine Learning (ML) algorithms within databases is a challenge due to the varied computational footprints of modern ML algorithms and the myriad of database technologies each with its own restrictive syntax. We introduce an Apache Spark-based micro-service orchestration framework that extends database operations to include web service primitives. Our system can orchestrate web services across hundreds of machines and takes full advantage of cluster, thread, and asynchronous parallelism. Using this framework, we provide large scale clients for intelligent services such as speech, vision, search, anomaly detection, and text analysis. This allows users to integrate ready-to-use intelligence into any datastore with an Apache Spark connector. To eliminate the majority of overhead from network communication, we also introduce a low-latency containerized version of our architecture. Finally, we demonstrate that the services we investigate are competitive on a variety of benchmarks, and present two applications of this framework to create intelligent search engines, and real-time auto race analytics systems.

Related papers

• BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data [61.936320820180875]
  Large language models (LLMs) have become increasingly pivotal across various domains. BabelBench is an innovative benchmark framework that evaluates the proficiency of LLMs in managing multimodal multistructured data with code execution. Our experimental findings on BabelBench indicate that even cutting-edge models like ChatGPT 4 exhibit substantial room for improvement.
  arXiv  Detail & Related papers  (2024-10-01T15:11:24Z)
• UQE: A Query Engine for Unstructured Databases [71.49289088592842]
  We investigate the potential of Large Language Models to enable unstructured data analytics. We propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections.
  arXiv  Detail & Related papers  (2024-06-23T06:58:55Z)
• GEqO: ML-Accelerated Semantic Equivalence Detection [3.5521901508676774]
  Common computation is crucial for efficient cluster resource utilization and reducing job execution time. detecting equivalence on large-scale analytics engines requires efficient and scalable solutions that are fully automated. We propose GEqO, a portable and lightweight machine-learning-based framework for efficiently identifying semantically equivalent computations at scale.
  arXiv  Detail & Related papers  (2024-01-02T16:37:42Z)
• CLAID: Closing the Loop on AI & Data Collection -- A Cross-Platform Transparent Computing Middleware Framework for Smart Edge-Cloud and Digital Biomarker Applications [2.953239144917]
  We present CLAID, an open-source framework based on transparent computing compatible with Android, iOS, WearOS, Linux, and Windows. We provide modules for data collection from various sensors as well as for the deployment of machine-learning models. We propose a novel methodology, "ML-Model in the Loop," for verifying deployed machine learning models.
  arXiv  Detail & Related papers  (2023-10-09T11:56:51Z)
• A comparison between traditional and Serverless technologies in a microservices setting [0.0]
  This study implements 9 prototypes of the same microservice application using different technologies. We use Amazon Web Services and start with an application that uses a more traditional deployment environment (Kubernetes) Migration to a serverless architecture is performed by combining and analysing the impact (both cost and performance) of the use of different technologies such as AWS ECS Fargate, AWS, DynamoDBDB.
  arXiv  Detail & Related papers  (2023-05-23T10:56:28Z)
• MOSAIC, acomparison framework for machine learning models [0.0]
  MOSAIC is a Python program for machine learning models. It makes implementing and testing arbitrary network architectures simpler, faster and less error-prone.
  arXiv  Detail & Related papers  (2023-01-30T15:29:24Z)
• Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training [80.03567604524268]
  Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. We propose SmartFL, a generic approach that optimize the server-side aggregation process. We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
  arXiv  Detail & Related papers  (2022-11-10T13:20:56Z)
• SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
  In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines. This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
  arXiv  Detail & Related papers  (2021-12-22T14:45:37Z)
• ESAI: Efficient Split Artificial Intelligence via Early Exiting Using Neural Architecture Search [6.316693022958222]
  Deep neural networks have been outperforming conventional machine learning algorithms in many computer vision-related tasks. The majority of devices are harnessing the cloud computing methodology in which outstanding deep learning models are responsible for analyzing the data on the server. In this paper, a new framework for deploying on IoT devices has been proposed which can take advantage of both the cloud and the on-device models.
  arXiv  Detail & Related papers  (2021-06-21T04:47:53Z)
• A Privacy-Preserving Distributed Architecture for Deep-Learning-as-a-Service [68.84245063902908]
  This paper introduces a novel distributed architecture for deep-learning-as-a-service. It is able to preserve the user sensitive data while providing Cloud-based machine and deep learning services.
  arXiv  Detail & Related papers  (2020-03-30T15:12:03Z)
• PyODDS: An End-to-end Outlier Detection System with Automated Machine Learning [55.32009000204512]
  We present PyODDS, an automated end-to-end Python system for Outlier Detection with Database Support. Specifically, we define the search space in the outlier detection pipeline, and produce a search strategy within the given search space. It also provides unified interfaces and visualizations for users with or without data science or machine learning background.
  arXiv  Detail & Related papers  (2020-03-12T03:30:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.