Complexity at Scale: A Quantitative Analysis of an Alibaba Microservice Deployment

• URL: http://arxiv.org/abs/2504.13141v2
• Date: Thu, 22 May 2025 13:55:49 GMT
• Title: Complexity at Scale: A Quantitative Analysis of an Alibaba Microservice Deployment
• Authors: Giles Winchester, George Parisis, Guoyao Xu, Luc Berthouze,
• Abstract summary: We analyse a microservice deployment dataset released by Alibaba.<n>We identify tens of thousands of characteristics that support an even broader array of front-end functionality.<n>We find that dependencies within the deployment at runtime can be different from the static view of the system.
• Score: 1.7124365853633132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent studies have begun to explore the characteristics of real-world large-scale microservice deployments. However, their operational complexities, and the degree to which these complexities are consistent across different deployments, remains under explored. In this paper, we analyse a microservice deployment dataset released by Alibaba to understand its scale, heterogeneity, and dynamicity, and compare our results to previous large-scale deployments to begin to understand their commonalities. We identify tens of thousands of microservices, that support an even broader array of front-end functionality. Moreover, our analysis shows wide-spread long-tailed distributions of characteristics between microservices, such as share of workload and dependencies, highlighting inequality. This diversity is also reflected in call graphs, with front-end service functionalities producing dominant and rarer, non-dominant, call graphs that can involve dissimilar microservice calls. We find that dependencies within the deployment at runtime can be different from the static view of the system, and that the deployment undergoes daily changes. We discuss the implications of our findings for state-of-the-art research in microservice management and research testbed realism.

Related papers

• Stochastic Encodings for Active Feature Acquisition [100.47043816019888]
  Active Feature Acquisition is an instance-wise, sequential decision making problem.<n>The aim is to dynamically select which feature to measure based on current observations, independently for each test instance.<n>Common approaches either use Reinforcement Learning, which experiences training difficulties, or greedily maximize the conditional mutual information of the label and unobserved features, which makes myopic.<n>We introduce a latent variable model, trained in a supervised manner. Acquisitions are made by reasoning about the features across many possible unobserved realizations in a latent space.
  arXiv  Detail & Related papers  (2025-08-03T23:48:46Z)
• Network Centrality as a New Perspective on Microservice Architecture [48.55946052680251]
  The adoption of Microservice Architecture has led to the identification of various patterns and anti-patterns, such as Nano/Mega/Hub services.<n>This study investigates whether centrality metrics (CMs) can provide new insights into MSA quality and facilitate the detection of architectural anti-patterns.
  arXiv  Detail & Related papers  (2025-01-23T10:13:57Z)
• Large Language Models as Realistic Microservice Trace Generators [54.85489678342595]
  Workload traces are essential to understand complex computer systems' behavior and manage processing and memory resources. This paper proposes a first-of-a-kind approach that relies on training a large language model to generate synthetic workload traces. Our model adapts to downstream trace-related tasks, such as predicting key trace features and infilling missing data.
  arXiv  Detail & Related papers  (2024-12-16T12:48:04Z)
• Microservices-based Software Systems Reengineering: State-of-the-Art and Future Directions [17.094721366340735]
  Designing software compatible with cloud-based Microservice Architectures (MSAs) is vital due to the performance, scalability, and availability limitations. We provide a comprehensive survey of current research into ways of identifying services in systems that can be redeployed as Static, dynamic, and hybrid approaches have been explored.
  arXiv  Detail & Related papers  (2024-07-18T21:59:05Z)
• CHASE: A Causal Heterogeneous Graph based Framework for Root Cause Analysis in Multimodal Microservice Systems [22.00860661894853]
  We propose a Causal Heterogeneous grAph baSed framEwork for root cause analysis, namely CHASE, for microservice systems with multimodal data. CHASE learns from the constructed hypergraph with hyperedges representing the flow of causality and performs root cause localization.
  arXiv  Detail & Related papers  (2024-06-28T07:46:51Z)
• Benchmarking Data Management Systems for Microservices [1.9948490148513414]
  Microservice architectures are a popular choice for deploying large-scale data-intensive applications. Existing microservice benchmarks lack essential data management challenges. Online Marketplace is a novel benchmark that embraces core data management requirements.
  arXiv  Detail & Related papers  (2024-05-19T11:55:45Z)
• VAMP: Visual Analytics for Microservices Performance [2.5824043688763543]
  Existing distributed tracing tools leverage swimlane as the primary means to support performance analysis. We introduce vamp once, the performance analysis of multiple end-to-end requests. We show how vamp aids in identifying RPC execution time deviations with significant impact on end-to-end performance.
  arXiv  Detail & Related papers  (2024-04-22T15:22:56Z)
• DIGIC: Domain Generalizable Imitation Learning by Causal Discovery [69.13526582209165]
  Causality has been combined with machine learning to produce robust representations for domain generalization. We make a different attempt by leveraging the demonstration data distribution to discover causal features for a domain generalizable policy. We design a novel framework, called DIGIC, to identify the causal features by finding the direct cause of the expert action from the demonstration data distribution.
  arXiv  Detail & Related papers  (2024-02-29T07:09:01Z)
• Migration to Microservices: A Comparative Study of Decomposition Strategies and Analysis Metrics [0.5076419064097734]
  We present a novel clustering method to identify potential in a given monolithic application. Our approach employs a density-based clustering algorithm considering static analysis, structural, and semantic relationships between classes.
  arXiv  Detail & Related papers  (2024-02-13T14:15:00Z)
• Twin Graph-based Anomaly Detection via Attentive Multi-Modal Learning for Microservice System [24.2074235652359]
  We propose MSTGAD, which seamlessly integrates all available data modalities via attentive multi-modal learning. We construct a transformer-based neural network with both spatial and temporal attention mechanisms to model the inter-correlations between different modalities. This enables us to detect anomalies automatically and accurately in real-time.
  arXiv  Detail & Related papers  (2023-10-07T06:28:41Z)
• Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentation [48.267001230607306]
  We propose a pretraining framework that enhances multiscale consistency in EM volumes. Our approach leverages a Siamese network architecture, integrating strong and weak data augmentations. It effectively captures voxel and feature consistency, showing promise for learning transferable representations for EM analysis.
  arXiv  Detail & Related papers  (2023-08-19T05:49:13Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanics [77.34726150561087]
  This work proposes a generative adversarial network tailored towards three-dimensional microstructure generation. The lightweight algorithm is able to learn the underlying properties of the material from a single microCT-scan without the need of explicit descriptors.
  arXiv  Detail & Related papers  (2022-05-31T13:26:51Z)
• Digital Fingerprinting of Microstructures [44.139970905896504]
  Finding efficient means of fingerprinting microstructural information is a critical step towards harnessing data-centric machine learning approaches. Here, we consider microstructure classification and utilise the resulting features over a range of related machine learning tasks. In particular, methods that leverage transfer learning with convolutional neural networks (CNNs), pretrained on the ImageNet dataset, are generally shown to outperform other methods.
  arXiv  Detail & Related papers  (2022-03-25T15:40:44Z)
• Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms [52.58031087639394]
  micro-expressions are involuntary and transient facial expressions. They can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, their detection and recognition is difficult and relies heavily on expert experiences.
  arXiv  Detail & Related papers  (2022-01-30T05:14:13Z)
• Learning Prototype-oriented Set Representations for Meta-Learning [85.19407183975802]
  Learning from set-structured data is a fundamental problem that has recently attracted increasing attention. This paper provides a novel optimal transport based way to improve existing summary networks. We further instantiate it to the cases of few-shot classification and implicit meta generative modeling.
  arXiv  Detail & Related papers  (2021-10-18T09:49:05Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective [69.99087941471882]
  We study the problem of supporting multiple machine vision analytics tasks with the compressed visual representation. By utilizing the intrinsic transferability among different tasks, our framework successfully constructs compact and expressive representations at low bit-rates. In order to impose compactness in the representations, we propose a codebook-based hyperprior.
  arXiv  Detail & Related papers  (2021-06-16T01:44:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.