ScalerEval: Automated and Consistent Evaluation Testbed for Auto-scalers in Microservices

• URL: http://arxiv.org/abs/2504.08308v1
• Date: Fri, 11 Apr 2025 07:18:53 GMT
• Title: ScalerEval: Automated and Consistent Evaluation Testbed for Auto-scalers in Microservices
• Abstract summary: ScalerEval is an end-to-end automated and consistent testbed for auto-scalers in researchers.<n>It integrates essential fundamental interfaces for implementation of auto-scalers and further orchestrates a one-click evaluation for researchers.
• Score: 7.4883371370599585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Auto-scaling is an automated approach that dynamically provisions resources for microservices to accommodate fluctuating workloads. Despite the introduction of many sophisticated auto-scaling algorithms, evaluating auto-scalers remains time-consuming and labor-intensive, as it requires the implementation of numerous fundamental interfaces, complex manual operations, and in-depth domain knowledge. Besides, frequent human intervention can inevitably introduce operational errors, leading to inconsistencies in the evaluation of different auto-scalers. To address these issues, we present ScalerEval, an end-to-end automated and consistent testbed for auto-scalers in microservices. ScalerEval integrates essential fundamental interfaces for implementation of auto-scalers and further orchestrates a one-click evaluation workflow for researchers. The source code is publicly available at \href{https://github.com/WHU-AISE/ScalerEval}{https://github.com/WHU-AISE/ScalerEval}.

Related papers

• Nautilus: From One Prompt to Plug-and-Play Robot Learning [32.048440678131676]
  NAUTILUS is an open-source harness that turns a single user prompt into ready-to-use reproduction, evaluation, and deployment.<n>It can not only automatically generate the required adapters and containers for existing implementations, but also wrap and onboard new or user-provided policies, simulators/benchmarks, and robots.<n>It aims to reduce the engineering burden of cross-family reproduction and evaluation in the ever-growing robot learning ecosystem.
  arXiv  Detail & Related papers  (2026-05-12T07:26:39Z)
• AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration [49.050688065974605]
  Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models.<n>We propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution.
  arXiv  Detail & Related papers  (2026-04-01T15:17:45Z)
• Human-AI Collaboration for Scaling Agile Regression Testing: An Agentic-AI Teammate from Manual to Automated Testing [1.4902707653987912]
  We present an agentic AI approach that generates system-level test scripts directly from validated specifications.<n>Our solution features a retrieval-augmented, multi-agent architecture integrated into Hacon's agile architecture.<n>Results show that the AI teammate significantly increases test script throughput and reduces manual authoring effort.
  arXiv  Detail & Related papers  (2026-03-09T10:19:13Z)
• SUNSET -- A Sensor-fUsioN based semantic SegmEnTation exemplar for ROS-based self-adaptation [2.8210523386413087]
  We present SUNSET, a ROS2-based exemplar that enables rigorous, repeatable evaluation of architecture-based self-adaptation.<n> SUNSET includes the segmentation pipeline, a trained ML model, uncertainty-injection scripts, a baseline controller, and step-by-step integration and evaluation documentation.
  arXiv  Detail & Related papers  (2026-01-20T08:40:57Z)
• AutoMetrics: Approximate Human Judgements with Automatically Generated Evaluators [57.003100107659684]
  AutoMetrics is a framework for synthesizing evaluation metrics under low-data constraints.<n>We show that AutoMetrics can be used as a proxy reward to equal effect as a verifiable reward.
  arXiv  Detail & Related papers  (2025-12-19T06:32:46Z)
• Extreme AutoML: Analysis of Classification, Regression, and NLP Performance [0.19791587637442667]
  Extreme Learning Machines (ELMs) use a fundamentally different type of neural architecture, producing better results at a significantly discounted computational cost. We benchmark the Extreme AutoML technology against Google's AutoML using several popular classification data sets from the University of California at Irvine's (UCI) repository.
  arXiv  Detail & Related papers  (2024-12-09T21:10:22Z)
• Position: A Call to Action for a Human-Centered AutoML Paradigm [83.78883610871867]
  Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) We argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems.
  arXiv  Detail & Related papers  (2024-06-05T15:05:24Z)
• AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning [54.47116888545878]
  AutoAct is an automatic agent learning framework for QA. It does not rely on large-scale annotated data and synthetic planning trajectories from closed-source models.
  arXiv  Detail & Related papers  (2024-01-10T16:57:24Z)
• GEqO: ML-Accelerated Semantic Equivalence Detection [3.5521901508676774]
  Common computation is crucial for efficient cluster resource utilization and reducing job execution time. detecting equivalence on large-scale analytics engines requires efficient and scalable solutions that are fully automated. We propose GEqO, a portable and lightweight machine-learning-based framework for efficiently identifying semantically equivalent computations at scale.
  arXiv  Detail & Related papers  (2024-01-02T16:37:42Z)
• Benchmarking Automated Machine Learning Methods for Price Forecasting Applications [58.720142291102135]
  We show the possibility of substituting manually created ML pipelines with automated machine learning (AutoML) solutions. Based on the CRISP-DM process, we split the manual ML pipeline into a machine learning and non-machine learning part. We show in a case study for the industrial use case of price forecasting, that domain knowledge combined with AutoML can weaken the dependence on ML experts.
  arXiv  Detail & Related papers  (2023-04-28T10:27:38Z)
• OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System [85.8338446357469]
  We introduce OmniForce, a human-centered AutoML system that yields both human-assisted ML and ML-assisted human techniques. We show how OmniForce can put an AutoML system into practice and build adaptive AI in open-environment scenarios.
  arXiv  Detail & Related papers  (2023-03-01T13:35:22Z)
• Towards Green Automated Machine Learning: Status Quo and Future Directions [71.86820260846369]
  AutoML is being criticised for its high resource consumption. This paper proposes Green AutoML, a paradigm to make the whole AutoML process more environmentally friendly.
  arXiv  Detail & Related papers  (2021-11-10T18:57:27Z)
• Induction and Exploitation of Subgoal Automata for Reinforcement Learning [75.55324974788475]
  We present ISA, an approach for learning and exploiting subgoals in episodic reinforcement learning (RL) tasks. ISA interleaves reinforcement learning with the induction of a subgoal automaton, an automaton whose edges are labeled by the task's subgoals. A subgoal automaton also consists of two special states: a state indicating the successful completion of the task, and a state indicating that the task has finished without succeeding.
  arXiv  Detail & Related papers  (2020-09-08T16:42:55Z)
• AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction [75.16836697734995]
  We propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS) AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service.
  arXiv  Detail & Related papers  (2020-03-25T06:53:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.