MIRAGE: Online LLM Simulation for Microservice Dependency Testing

• URL: http://arxiv.org/abs/2604.04806v3
• Date: Sun, 12 Apr 2026 05:46:58 GMT
• Title: MIRAGE: Online LLM Simulation for Microservice Dependency Testing
• Abstract summary: Existing approaches to microservice dependency simulation generate static artifacts before test execution.<n>We propose online LLM simulation, a runtime approach where the LLM answers each dependency request as it arrives.<n>We instantiate this approach in MIRAGE and evaluate it on 110 test scenarios across three microservice systems.
• Score: 5.94231111588812
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing approaches to microservice dependency simulation--record-replay, pattern-mining, and specification-driven stubs--generate static artifacts before test execution. These artifacts can only reproduce behaviors encoded at generation time; on error-handling and code-reasoning scenarios, which are underrepresented in typical trace corpora, record-replay achieves 0% and 12% fidelity in our evaluation. We propose online LLM simulation, a runtime approach where the LLM answers each dependency request as it arrives, maintaining cross-request state throughout a test scenario. The model reads the dependency's source code, caller code, and production traces, then simulates behavior on demand--trading latency (~3 s per request) and cost ($0.16-$0.82 per dependency) for coverage on scenarios that static artifacts miss. We instantiate this approach in MIRAGE and evaluate it on 110 test scenarios across three microservice systems (Google's Online Boutique, Weaveworks' Sock Shop, and a custom system). In white-box mode, MIRAGE achieves 99% status-code and 99% response-shape fidelity, compared to 62% / 16% for record-replay. A signal ablation shows dependency source code is often sufficient (100% alone); without it, the model retains error-code accuracy (94%) but loses response-structure fidelity (75%). Results are stable across three LLM families (within 3%) and deterministic across repeated runs. Caller integration tests produce the same pass/fail outcomes with MIRAGE as with real dependencies (8/8 scenarios).

Related papers

• From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix [55.08143827481755]
  We consolidate traffic from over 200 internal applications onto a single model.<n>We train a separate GRPO expert per axis and merge them via two-stage SLERP.<n>The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.
  arXiv  Detail & Related papers  (2026-09-01T17:39:26Z)
• LMSM: LLM Security Framework Inspired by Linux Security Modules [56.93644694627783]
  Large language models (LLMs) are increasingly deployed with layered defenses, yet malicious prompts can still bypass them.<n>We present Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving.
  arXiv  Detail & Related papers  (2026-08-26T12:13:05Z)
• LLM-Guided Heuristic Design from Simulation Traces: A Case Study in Dynamic Production and AGV Scheduling [10.289256706393084]
  Simulation-based optimization (SBO) evaluates executable policies under dynamics.<n>Most methods treat the simulator as a black box: aggregate scores rank candidates without revealing why they fail or which policy logic should change.<n>We present an LLM-guided design framework that uses repeated simulation for selection and event-level traces for diagnosis.
  arXiv  Detail & Related papers  (2026-08-10T09:20:56Z)
• A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination [51.06539604709775]
  We propose A-SR, a self-evolving agentic framework that shifts the control unit from expression edits to role-conditioned evidence views.<n>A-SR coordinates formula discovery through routing among coordination protocols, an online evaluator-reward role policy, and state-routed process memory.
  arXiv  Detail & Related papers  (2026-08-05T14:01:10Z)
• TraceCompiler: Skill-Guided Mining and Compilation of LLM Agent Traces into Mostly Deterministic Workflows [2.435006380732194]
  We present TraceCompiler, a skill-guided system that compiles noisy traces into trajectories.<n>Every hard edge carries an auditable evidence, and ambiguous relations are marked suspected.<n>We measure call reduction but not offline compilation cost, so we claim no net efficiency result.
  arXiv  Detail & Related papers  (2026-08-03T01:05:06Z)
• Industrial Practice of LLM-Based Test Case Carving and Assertion Generation (Experience Paper) [14.766370287473519]
  This paper presents NL2Test, an end-to-end approach and tool that generates executable API regression tests from natural-language scenarios.<n> NL2Test addresses two coupled tasks: test case carving and assertion generation.<n>We evaluate NL2Test on 51 industrial regression scenarios extracted from a large consumer-facing Internet company.
  arXiv  Detail & Related papers  (2026-07-27T04:52:51Z)
• MEME: Multi-entity & Evolving Memory Evaluation [76.57263966646404]
  MEME defines six tasks spanning the full space defined by the multi-entity and evolving axes.<n> evaluating six memory systems spanning three memory paradigms on 100 controlled episodes.<n>We find that all systems collapse on dependency reasoning under the default configuration.
  arXiv  Detail & Related papers  (2026-05-12T17:55:10Z)
• ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox [61.862814740220806]
  $textbfComplexMCP$ is a benchmark designed to evaluate agents in rigorous conditions.<n>Built on the Model Context Protocol (MCP), $textbfComplexMCP$ provides over 300 meticulously tested tools.
  arXiv  Detail & Related papers  (2026-05-11T16:20:51Z)
• LLM Readiness Harness: Evaluation, Observability, and CI Gates for LLM/RAG Applications [51.56484100374058]
  We present a readiness harness for LLM and RAG applications that turns evaluation into a deployment decision workflow.<n>The system combines automated benchmarks, OpenTelemetry observability, and CI quality gates under a minimal API contract.<n>We evaluate the harness on ticket-routing and BEIR tasks with full Azure matrix coverage.
  arXiv  Detail & Related papers  (2026-03-28T18:03:32Z)
• Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards [76.49428173793386]
  LLMs must invoke multiple dependent APIs in the correct order while propagating intermediate outputs.<n>Training models to handle such faces two obstacles: existing environments focus on simple per-turn function calls with simulated data, and binary rewards.<n>We present a framework addressing both challenges. First, we construct a reinforcement learning environment backed by a large-scale cache of real API responses, enabling a data synthesis pipeline that samples valid multi-step orchestration traces.<n>Second, we propose a graduated reward design that decomposes correctness into atomic validity and orchestration.
  arXiv  Detail & Related papers  (2026-03-25T18:31:39Z)
• dVoting: Fast Voting for dLLMs [71.572316901001]
  Diffusion Large Language Models (dLLMs) represent a new paradigm beyond autoregressive modeling.<n>dLLMs can generate tokens at arbitrary positions in parallel, endowing them with significant potential for parallel test-time scaling.<n>We introduce dVoting, a fast voting technique that boosts reasoning capability without training, with only an acceptable extra computational overhead.
  arXiv  Detail & Related papers  (2026-02-12T16:35:05Z)
• Evaluating Asynchronous Semantics in Trace-Discovered Resilience Models: A Case Study on the OpenTelemetry Demo [0.0]
  We revisit a trace-discovered connectivity model that derives a service dependency graph from traces.<n>We use Monte Carlo simulation to estimate endpoint availability under fail-stop service failures.
  arXiv  Detail & Related papers  (2025-12-13T13:08:05Z)
• PALM: Synergizing Program Analysis and LLMs to Enhance Rust Unit Test Coverage [14.702182387149547]
  This paper presents PALM, an approach that leverages large language models (LLMs) to enhance the generation of high-coverage unit tests.<n> PALM performs program analysis to identify branching conditions within functions, which are then combined into path constraints.<n>We implement the approach and evaluate it in 15 open-source Rust crates.
  arXiv  Detail & Related papers  (2025-06-10T17:21:21Z)
• Sample, Don't Search: Rethinking Test-Time Alignment for Language Models [55.2480439325792]
  We introduce QAlign, a new test-time alignment approach.<n>As we scale test-time compute, QAlign converges to sampling from the optimal aligned distribution for each individual prompt.<n>By adopting recent advances in Markov chain Monte Carlo for text generation, our method enables better-aligned outputs without modifying the underlying model or even requiring logit access.
  arXiv  Detail & Related papers  (2025-04-04T00:41:40Z)
• Reinforcement Learning for Long-Horizon Interactive LLM Agents [56.9860859585028]
  Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests.<n>We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments.<n>We derive LOOP, a data- and memory-efficient variant of proximal policy optimization.
  arXiv  Detail & Related papers  (2025-02-03T18:35:42Z)
• An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation [3.9762912548964864]
  This paper presents a large-scale empirical evaluation on the effectiveness of Large Language Models for automated unit test generation. We implement our approach in TestPilot, a test generation tool for JavaScript that automatically generates unit tests for all API functions in an npm package. We find that 92.8% of TestPilot's generated tests have no more than 50% similarity with existing tests.
  arXiv  Detail & Related papers  (2023-02-13T17:13:41Z)
• Mimicking Production Behavior with Generated Mocks [11.367562045401554]
  We propose to monitor an application in production to generate tests that mimic realistic execution scenarios through mocks. The approach is automated and implemented in an open-source tool called RICK. All the generated test cases are executable, and 52.4% of them successfully mimic the complete execution context of the target methods observed in production.
  arXiv  Detail & Related papers  (2022-08-02T09:08:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.