VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?

• URL: http://arxiv.org/abs/2605.06068v1
• Date: Thu, 07 May 2026 11:54:53 GMT
• Title: VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
• Abstract summary: We propose VibeServe, the first agentic loop that generates entire LLM serving stacks end-to-end.<n>In the standard deployment setting, where existing stacks are highly optimized, VibeServe remains competitive with vLLM.<n>In non-standard scenarios, VibeServe outperforms existing systems by exploiting opportunities that generic systems miss.
• Score: 7.395579663607589
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: For years, we have built LLM serving systems like any other critical infrastructure: a single general-purpose stack, hand-tuned over many engineer-years, meant to support every model and workload. In this paper, we take the opposite bet: a multi-agent loop that automatically synthesizes bespoke serving systems for different usage scenarios. We propose VibeServe, the first agentic loop that generates entire LLM serving stacks end-to-end. VibeServe uses an outer loop to plan and track the search over system designs, and an inner loop to implement candidates, check correctness, and measure performance on the target benchmark. In the standard deployment setting, where existing stacks are highly optimized, VibeServe remains competitive with vLLM, showing that generation-time specialization need not come at the cost of performance. More interestingly, in non-standard scenarios, VibeServe outperforms existing systems by exploiting opportunities that generic systems miss in six scenarios involving non-standard model architectures, workload knowledge, and hardware-specific optimizations. Together, these results suggest a different point in the design space for infrastructure software: generation-time specialization rather than runtime generality. Code is available at https://github.com/uw-syfi/vibe-serve.

Related papers

• The Time is Here for Just-in-Time Systems: Challenges and Opportunities [102.80383421480155]
  Core systems like key-value stores have historically taken years to build, and are designed to be general so as to amortize cost across deployments.<n>We argue that LLM-based coding agents now make a different approach tractable: Just-in-Time Systems.<n>We present a JIT system synthesis pipeline, Jitskit, and explore its effectiveness in synthesizing key-value stores from spec cards.
  arXiv  Detail & Related papers  (2026-05-22T18:03:41Z)
• StarVLA-$α$: Reducing Complexity in Vision-Language-Action Systems [89.12403322299963]
  StarVLA-$$ is a simple yet strong baseline to study VLA design choices under controlled conditions.<n>We re-evaluate several key design axes, including action modeling strategies, robot-specific pretraining, and interface engineering.<n>Our single generalist model outperforms $_0.5$ by 20% on the public real-world RoboChallenge benchmark.
  arXiv  Detail & Related papers  (2026-04-13T17:30:01Z)
• MASEval: Extending Multi-Agent Evaluation from Models to Systems [77.09381093359598]
  MASEval is a framework-agnostic library that treats the entire system as the unit of analysis.<n>It allows researchers to explore all components of agentic systems, opening new avenues for principled system design.
  arXiv  Detail & Related papers  (2026-03-09T18:46:17Z)
• AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving [16.664502126572856]
  AIConfigurator is a unified performance-modeling system for Large Language Model (LLM) inference.<n>It enables rapid, framework-a configuration search without requiring GPU-based profiling.<n>It identifies superior serving configurations that improve performance by up to 40% for dense models.
  arXiv  Detail & Related papers  (2026-01-09T20:03:57Z)
• EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis [101.67583081810136]
  Large language models (LLMs) are expected to be trained to act as agents in various real-world environments.<n>This process relies on rich and varied tool-interaction sandboxes.<n>We propose EnvScaler, an automated framework for scalable tool-interaction environments.
  arXiv  Detail & Related papers  (2026-01-09T14:32:06Z)
• Optimizing PyTorch Inference with LLM-Based Multi-Agent Systems [1.2289544895833646]
  We present a framework for comparing multi-agent PyTorch optimization systems.<n>We show that exploit-heavy strategies perform best when paired with error-fixing agents.<n>The best implementation achieves an average 2.88x speedup on an H100 GPU.
  arXiv  Detail & Related papers  (2025-11-21T05:37:38Z)
• UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist [107.04196084992907]
  We introduce UniVA, an omni-capable multi-agent framework for next-generation video generalists.<n>UniVA employs a Plan-and-Act dual-agent architecture that drives a highly automated and proactive workflow.<n>We also introduce UniVA-Bench, a benchmark suite of multi-step video tasks spanning understanding, editing, segmentation, and generation.
  arXiv  Detail & Related papers  (2025-11-11T17:58:13Z)
• Towards Online Code Specialization of Systems [1.470243173272826]
  We advocate for a radically different approach for performance-critical low-level systems.<n>We leverage just-in-time compilation to change systems code based on developer-specified specialization points.<n>We demonstrate that online specialization (i) is feasible even for low-level systems code, such as network stacks, (ii) improves system performance without the need for complex cost models, (iii) incurs low developer effort, especially compared to manual exploration.
  arXiv  Detail & Related papers  (2025-01-20T09:55:15Z)
• Multi-Objective Deep Reinforcement Learning for Optimisation in Autonomous Systems [3.2826250607043796]
  Multi-Objective Reinforcement Learning (MORL) techniques exist but they have mostly been applied in RL benchmarks rather than real-world AS systems. In this work, we use a MORL technique called Deep W-Learning (DWN) to find the optimal configuration for runtime performance optimization. We compare DWN to two single-objective optimization implementations: epsilon-greedy algorithm and Deep Q-Networks.
  arXiv  Detail & Related papers  (2024-08-02T11:16:09Z)
• Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models [8.02264001053969]
  Large language models (LLMs) have shown remarkable performance across a wide range of applications, often outperforming human experts.<n>With constant innovation in LLM serving optimizations and model architecture evolving at breakneck speed, the hardware requirements to meet Service Level Objectives (SLOs) remain an open research question.<n>We present an analytical tool, GenZ, to efficiently navigate the relationship between diverse LLM model architectures and AI platform design parameters.
  arXiv  Detail & Related papers  (2024-06-03T18:00:50Z)
• Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models [55.5610165938949]
  Fine-tuning vision-language models (VLMs) has gained increasing popularity due to its practical value. This paper explores the collaborative potential of leveraging much weaker VLMs to enhance the generalization of a robust single model. We introduce three customized ensemble strategies, each tailored to one specific scenario. The proposed ensemble strategies are evaluated on zero-shot, base-to-new, and cross-dataset generalization, achieving new state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-11-28T05:17:25Z)
• Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors [44.5740422079]
  We show that pretraining with standard denoising objectives leads to dramatic gains across multiple architectures. In stark contrast to prior works, we find vanilla Transformers to match the performance of S4 on Long Range Arena when properly pretrained.
  arXiv  Detail & Related papers  (2023-10-04T17:17:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.