Revealing Performance Issues in Server-side WebAssembly Runtimes via Differential Testing

• URL: http://arxiv.org/abs/2309.12167v1
• Date: Thu, 21 Sep 2023 15:25:18 GMT
• Title: Revealing Performance Issues in Server-side WebAssembly Runtimes via Differential Testing
• Abstract summary: We design a novel differential testing approach WarpDiff to identify performance issues in server-side Wasm runtimes. We identify abnormal cases where the execution time ratio significantly deviates from the oracle ratio and locate the Wasm runtimes that cause the performance issues.
• Score: 28.187405253760687
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: WebAssembly (Wasm) is a bytecode format originally serving as a compilation target for Web applications. It has recently been used increasingly on the server side, e.g., providing a safer, faster, and more portable alternative to Linux containers. With the popularity of server-side Wasm applications, it is essential to study performance issues (i.e., abnormal latency) in Wasm runtimes, as they may cause a significant impact on server-side applications. However, there is still a lack of attention to performance issues in server-side Wasm runtimes. In this paper, we design a novel differential testing approach WarpDiff to identify performance issues in server-side Wasm runtimes. The key insight is that in normal cases, the execution time of the same test case on different Wasm runtimes should follow an oracle ratio. We identify abnormal cases where the execution time ratio significantly deviates from the oracle ratio and subsequently locate the Wasm runtimes that cause the performance issues. We apply WarpDiff to test five popular server-side Wasm runtimes using 123 test cases from the LLVM test suite and demonstrate the top 10 abnormal cases we identified. We further conduct an in-depth analysis of these abnormal cases and summarize seven performance issues, all of which have been confirmed by the developers. We hope our work can inspire future investigation on improving Wasm runtime implementation and thus promoting the development of server-side Wasm applications.

Related papers

• LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues [80.29362825271768]
  LongMemEval-V2 is a benchmark for evaluating whether memory systems can help agents acquire the experience needed to become knowledgeable colleagues in customized environments.<n>LME-V2 contains 451 manually curated questions covering five core memory abilities for web agents.<n>We propose a suite of two memory methods: AgentRunbook-R, an efficient RAG-based memory with knowledge pools for raw state observations, events, and strategy notes, and AgentRunbook-C, which stores trajectories as files and invokes a coding agent to gather evidence in an augmented sandbox.
  arXiv  Detail & Related papers  (2026-05-12T17:59:34Z)
• Debugging Performance Issues in WebAssembly Runtimes via Mutation-based Inference [5.468106603770207]
  We present WarpL, a mutation-based approach that aims to identify the exact suboptimal instruction sequences responsible for the performance issues in Wasm runtimes.<n>We implement WarpL as an open-source tool and evaluate it on 12 real-world performance issues across three widely used Wasm runtimes.
  arXiv  Detail & Related papers  (2026-04-15T10:16:28Z)
• WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing [57.7131457251794]
  We introduce WebTestBench, a benchmark for evaluating end-to-end automated web testing.<n>We decompose the testing process into two cascaded sub-tasks, checklist generation and defect detection, and propose WebTester.<n>Our findings expose a substantial gap between current computer-use agent capabilities and industrial-grade deployment demands.
  arXiv  Detail & Related papers  (2026-03-26T09:27:29Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• SafeLoad: Efficient Admission Control Framework for Identifying Memory-Overloading Queries in Cloud Data Warehouses [59.68732483257323]
  Memory overload is a common form of resource exhaustion in cloud data warehouses.<n>We propose SafeLoad, the first query admission control framework specifically designed to identify memory-overloading (MO) queries.<n>We show that SafeLoad achieves state-of-the-art prediction performance with low online and offline time overhead.
  arXiv  Detail & Related papers  (2026-01-05T08:29:51Z)
• Screencast-Based Analysis of User-Perceived GUI Responsiveness [53.53923672866705]
  tool is a technique that measures GUI responsiveness directly from mobile screencasts.<n>It uses computer vision to detect user interactions and analyzes frame-level visual changes to compute two key metrics.<n>tool has been deployed in an industrial testing pipeline and analyzes thousands of screencasts daily.
  arXiv  Detail & Related papers  (2025-08-02T12:13:50Z)
• When Should I Run My Application Benchmark?: Studying Cloud Performance Variability for the Case of Stream Processing Applications [1.3398445165628463]
  This paper empirically quantify the impact of cloud performance variability on benchmarking results. With approximately 591 hours of experiments, deploying 789 clusters on AWS and executing 2366 benchmarks, this is likely the largest study of its kind.
  arXiv  Detail & Related papers  (2025-04-16T07:22:44Z)
• Distinguishability-guided Test Program Generation for WebAssembly Runtime Performance Testing [28.920256869194315]
  High performance is a critical design goal of WebAssembly (Wasm) Research on Wasm runtime performance testing still suffers from insufficient high-quality test programs. In particular, WarpGen has identified seven new performance issues in three Wasm runtimes.
  arXiv  Detail & Related papers  (2024-12-28T09:51:23Z)
• SeBS-Flow: Benchmarking Serverless Cloud Function Workflows [51.4200085836966]
  We propose the first serverless workflow benchmarking suite SeBS-Flow. SeBS-Flow includes six real-world application benchmarks and four microbenchmarks representing different computational patterns. We conduct comprehensive evaluations on three major cloud platforms, assessing performance, cost, scalability, and runtime deviations.
  arXiv  Detail & Related papers  (2024-10-04T14:52:18Z)
• An Effectively $Ω(c)$ Language and Runtime [0.0]
  Good performance of an application is conceptually more of a binary function. Our vision is to create a language and runtime that is designed to be $Omega(c)$ in its performance.
  arXiv  Detail & Related papers  (2024-09-30T16:57:45Z)
• eWAPA: An eBPF-based WASI Performance Analysis Framework for WebAssembly Runtimes [3.804314901623159]
  WebAssembly (Wasm) is a low-level bytecode format that can run in modern browsers. We propose an eBPF-based WASI performance analysis framework. It collects key performance metrics of the runtime under different I/O load conditions, such as total execution time, startup time, WASI execution time, and syscall time.
  arXiv  Detail & Related papers  (2024-09-16T13:03:09Z)
• Research on WebAssembly Runtimes: A Survey [22.031129110987017]
  WebAssembly (abbreviated as Wasm) was initially introduced for the Web but quickly extended its reach into various domains beyond the Web. This paper provides a comprehensive survey of research on WebAssembly runtimes.
  arXiv  Detail & Related papers  (2024-04-19T04:36:38Z)
• Green AI: A Preliminary Empirical Study on Energy Consumption in DL Models Across Different Runtime Infrastructures [56.200335252600354]
  It is common practice to deploy pre-trained models on environments distinct from their native development settings. This led to the introduction of interchange formats such as ONNX, which includes its infrastructure, and ONNX, which work as standard formats.
  arXiv  Detail & Related papers  (2024-02-21T09:18:44Z)
• WRTester: Differential Testing of WebAssembly Runtimes via Semantic-aware Binary Generation [19.78427170624683]
  We present WRTester, a novel differential testing framework that can generated complicated Wasm test cases by disassembling and assembling real-world Wasm binaries. For further pinpointing the root causes of unexpected behaviors, we design a runtime-agnostic root cause location method to accurately locate bugs. We have uncovered 33 unique bugs in popular Wasm runtimes, among which 25 have been confirmed.
  arXiv  Detail & Related papers  (2023-12-16T14:02:42Z)
• Overload: Latency Attacks on Object Detection for Edge Devices [47.9744734181236]
  This paper investigates latency attacks on deep learning applications. Unlike common adversarial attacks for misclassification, the goal of latency attacks is to increase the inference time. We use object detection to demonstrate how such kind of attacks work.
  arXiv  Detail & Related papers  (2023-04-11T17:24:31Z)
• Automatic Cause Detection of Performance Problems in Web Applications [1.749935196721634]
  We propose a method of extracting the internal behavior of web requests and introduce a pipeline that detects performance issues in web requests. Experiments revealed that this pipeline is indeed able to detect slow web requests and provide additional insights into their true root causes.
  arXiv  Detail & Related papers  (2021-03-08T18:17:40Z)
• Emerging App Issue Identification via Online Joint Sentiment-Topic Tracing [66.57888248681303]
  We propose a novel emerging issue detection approach named MERIT. Based on the AOBST model, we infer the topics negatively reflected in user reviews for one app version. Experiments on popular apps from Google Play and Apple's App Store demonstrate the effectiveness of MERIT.
  arXiv  Detail & Related papers  (2020-08-23T06:34:05Z)
• Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Framework [68.96770035057716]
  A/B testing is a business strategy to compare a new product with an old one in pharmaceutical, technological, and traditional industries. This paper introduces a reinforcement learning framework for carrying A/B testing in online experiments.
  arXiv  Detail & Related papers  (2020-02-05T10:25:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.