A Benchmark for Data Management in Microservices

• URL: http://arxiv.org/abs/2403.12605v2
• Date: Sun, 19 May 2024 11:41:11 GMT
• Title: A Benchmark for Data Management in Microservices
• Authors: Rodrigo Laigner, Zhexiang Zhang, Yijian Liu, Leonardo Freitas Gomes, Yongluan Zhou,
• Abstract summary: We present Online Marketplace, a microservice benchmark that incorporates core data management challenges. These challenges include transaction processing, query processing, event processing, constraint enforcement, and data replication. We present the challenges we faced in creating workloads that accurately reflect the state-of-the-art data platforms.
• Score: 1.9338699922911442
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Microservice architectures emerged as a popular architecture for designing scalable distributed applications. Although microservices have been extensively employed in industry settings for over a decade, there is little understanding of the data management challenges that arise in these applications. As a result, it is difficult to advance data system technologies for supporting microservice applications. To fill this gap, we present Online Marketplace, a microservice benchmark that incorporates core data management challenges that existing benchmarks have not sufficiently addressed. These challenges include transaction processing, query processing, event processing, constraint enforcement, and data replication. We have defined criteria for various data management issues to enable proper comparison across data systems and platforms. After specifying the benchmark, we present the challenges we faced in creating workloads that accurately reflect the dynamic state of the microservices. We also discuss issues that we encountered when implementing Online Marketplace in state-of-the-art data platforms and meeting the criteria. Our evaluation demonstrates that the benchmark is a valuable tool for testing important properties sought by microservice practitioners. As a result, our proposed benchmark will facilitate the design of future data systems to meet the expectations of microservice practitioners.

Related papers

• BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data [61.936320820180875]
  Large language models (LLMs) have become increasingly pivotal across various domains. BabelBench is an innovative benchmark framework that evaluates the proficiency of LLMs in managing multimodal multistructured data with code execution. Our experimental findings on BabelBench indicate that even cutting-edge models like ChatGPT 4 exhibit substantial room for improvement.
  arXiv  Detail & Related papers  (2024-10-01T15:11:24Z)
• Insights on Microservice Architecture Through the Eyes of Industry Practitioners [39.58317527488534]
  The adoption of microservice architecture has seen a considerable upswing in recent years. This study investigates the motivations, activities, and challenges associated with migrating from monolithic legacy systems.
  arXiv  Detail & Related papers  (2024-08-19T21:56:58Z)
• An Empirical Study on Challenges of Event Management in Microservice Architectures [3.0184596495288263]
  This paper provides the first comprehensive characterization of event management practices and challenges. We find that developers encounter many problems, including large event payloads, auditing event flows, and ordering constraints processing events. This suggests that developers are not sufficiently served by stateof-the-practice technologies.
  arXiv  Detail & Related papers  (2024-08-01T10:19:37Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)
• Benchmarking Data Management Systems for Microservices [1.9948490148513414]
  Microservice architectures are a popular choice for deploying large-scale data-intensive applications. Existing microservice benchmarks lack essential data management challenges. Online Marketplace is a novel benchmark that embraces core data management requirements.
  arXiv  Detail & Related papers  (2024-05-19T11:55:45Z)
• A microservice architecture for real-time IoT data processing: A reusable Web of things approach for smart ports [4.612539452170667]
  We propose a fully reusable microservice architecture, standardized through the use of the Web of things paradigm. We present a fully reusable implementation of the architecture in the field of air quality monitoring and alerting smart ports.
  arXiv  Detail & Related papers  (2024-01-27T11:40:38Z)
• Data Acquisition: A New Frontier in Data-centric AI [65.90972015426274]
  We first present an investigation of current data marketplaces, revealing lack of platforms offering detailed information about datasets. We then introduce the DAM challenge, a benchmark to model the interaction between the data providers and acquirers. Our evaluation of the submitted strategies underlines the need for effective data acquisition strategies in Machine Learning.
  arXiv  Detail & Related papers  (2023-11-22T22:15:17Z)
• Benchmarks for End-to-End Microservices Testing [2.6245844272542027]
  We created a test benchmark containing full functional testing coverage for two well-established open-source microservice systems. We also conducted a case study to identify the best approaches to take to validate a full coverage of tests.
  arXiv  Detail & Related papers  (2023-06-09T13:42:53Z)
• DataPerf: Benchmarks for Data-Centric AI Development [81.03754002516862]
  DataPerf is a community-led benchmark suite for evaluating ML datasets and data-centric algorithms. We provide an open, online platform with multiple rounds of challenges to support this iterative development. The benchmarks, online evaluation platform, and baseline implementations are open source.
  arXiv  Detail & Related papers  (2022-07-20T17:47:54Z)
• SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
  In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines. This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
  arXiv  Detail & Related papers  (2021-12-22T14:45:37Z)
• Benchmarks for Deep Off-Policy Evaluation [152.28569758144022]
  We present a collection of policies that can be used for benchmarking off-policy evaluation. The goal of our benchmark is to provide a standardized measure of progress that is motivated from a set of principles. We provide open-source access to our data and code to foster future research in this area.
  arXiv  Detail & Related papers  (2021-03-30T18:09:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.