DiLaServe: High SLO Attainment Serving for Diffusion Language Models

• URL: http://arxiv.org/abs/2606.29094v1
• Date: Sat, 27 Jun 2026 21:21:53 GMT
• Title: DiLaServe: High SLO Attainment Serving for Diffusion Language Models
• Abstract summary: Diffusion language models (DLMs) offer higher inference throughput while maintaining competitive quality.<n>We present DiLaServe, a cluster-level serving system for DLMs.<n>Across multiple benchmarks and real-world traces, DiLaServe improves SLO attainment by up to 56.6 percentage points and reduces end-to-end request latency by up to 46%.
• Score: 1.840765166456233
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion language models (DLMs) have recently emerged as a promising alternative to conventional autoregressive language models. By generating multiple tokens in parallel during each denoising step, they offer higher inference throughput while maintaining competitive quality. However, realizing these throughput gains while meeting latency SLOs in a serving system requires addressing challenges introduced by DLMs' unique characteristics. These include navigating the speed-quality tradeoff created by confidence-based denoising, choosing appropriate parallelization levels across model instances under fluctuating load, and coordinating approximate KV caching mechanisms that introduce non-uniform per-step costs. To address these challenges, we present DiLaServe, a cluster-level serving system for DLMs. DiLaServe enables deadline-aware scheduling and adaptive load control through confidence-threshold adjustment, and dynamically reconfigures the cluster by solving a quality-aware optimization problem, while explicitly modeling the step-level heterogeneity introduced by approximate KV caching. Across multiple benchmarks and real-world traces, DiLaServe improves SLO attainment by up to 56.6 percentage points and reduces end-to-end request latency by up to 46\% while incurring less than 1\% accuracy drop.

Related papers

• End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Services [13.04369901641143]
  Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference.<n>We develop a cross-slot inference model that captures transmission, prefill, redistribution-level decoding, and key-value cache evolution for each diverse request.<n>We propose the LYREO approach that transforms the long-term load-balancing constraint via Lyapunov optimization and employs reward with sequencebased return prediction.
  arXiv  Detail & Related papers  (2026-09-15T13:50:54Z)
• Diffusion Language Models for Mobile Edge Agentic AI: Foundations, Applications, and Challenges [58.887152635127165]
  Diffusion language models (DLMs) offer a non-autoregressive alternative for mobile edge agentic artificial intelligence (AI)<n>DLMs can update multiple uncertain tokens in parallel and exploit bidirectional context throughout the generation process.<n>This survey reviews DLM foundations and analyzes their suitability for edge settings under latency, memory, energy, bandwidth, privacy, and reliability constraints.
  arXiv  Detail & Related papers  (2026-09-04T06:16:49Z)
• FLARE: Diffusion for Hybrid Language Model [72.60770374799634]
  FLARE is a systematic conversion framework for hybrid-attention large language models.<n>It combines a token-equal AR-and-diffusion objective, hardware-aware kernels, and unified inference, enabling one checkpoint to support both AR-style verified decoding and diffusion-style parallel denoising.<n>Our results suggest practical dLLMs are limited not only by decoding algorithms, but also by transfer data quality and the training inefficiency of current block-diffusion objectives.
  arXiv  Detail & Related papers  (2026-06-01T06:58:15Z)
• AsynDBT: Asynchronous Distributed Bilevel Tuning for efficient In-Context Learning with Large Language Models [4.4866154758274375]
  In-context learning (ICL) has emerged as a promising paradigm that enables LLMs to adapt to new tasks using examples provided within the input.<n>Previous FL approaches that incorporate ICL have struggled with severe straggler problems and challenges associated with heterogeneous non-identically data.<n>We propose an asynchronous distributed bilevel tuning (AsynDBT) algorithm that optimize both in-context learning samples and prompt fragments based on the feedback from the LLM.
  arXiv  Detail & Related papers  (2026-02-06T13:07:49Z)
• CALM: A Self-Adaptive Orchestration Approach for QoS-Aware Routing in Small Language Model based Systems [0.6999740786886536]
  CALM is a self-adaptive orchestration mechanism based on MAPE-K.<n>It reduces latency by approximately 40% and energy consumption by 50%.<n>Our evaluation shows that CALM reduces latency by approximately 40% and energy consumption by 50%.
  arXiv  Detail & Related papers  (2026-02-03T15:20:14Z)
• CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems [62.24576366776727]
  We propose a latency-aware scheduling framework to minimize total inference latency.<n>We show that the proposed method significantly reduces cold-start latency compared to baseline strategies.
  arXiv  Detail & Related papers  (2025-08-15T07:49:22Z)
• Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission [87.68447072141402]
  Hybrid Language Models (HLMs) combine the low-latency efficiency of Small Language Models (SLMs) on edge devices with the high accuracy of Large Language Models (LLMs) on centralized servers.<n>We propose FedHLM, a communication-efficient HLM framework that integrates uncertainty-aware inference with Federated Learning (FL)
  arXiv  Detail & Related papers  (2025-06-30T02:56:11Z)
• Adaptive Request Scheduling for CodeLLM Serving with SLA Guarantees [6.110847503516972]
  Existing Large Language Models (CodeMs) are increasingly integrated into modern software development.<n>Yet, self-hosted environments remain a significant challenge in resource-constrained serving environments.<n>We propose SABER, a dynamic strategy that predicts per-request SLA feasibility and decisions in real time.<n>Our results demonstrate that SLA-aware, adaptive scheduling is key to robust, high-performance CodeLL serving.
  arXiv  Detail & Related papers  (2025-06-24T14:44:33Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding [51.711605076319216]
  Diffusion-based large language models (Diffusion LLMs) have shown promise for non-autoregressive text generation with parallel decoding capabilities.<n>We introduce a novel block-wise approximate KV Cache mechanism tailored for bidirectional diffusion models, enabling cache reuse with negligible performance drop.<n>We propose a confidence-aware parallel decoding strategy that selectively decodes tokens exceeding a confidence threshold, mitigating dependency violations and maintaining generation quality.
  arXiv  Detail & Related papers  (2025-05-28T17:39:15Z)
• AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding [12.106234303559571]
  We present AdaServe, the first serving system designed to support efficient multi-SLO serving through SLO-customized speculative decoding.<n>AdaServe formulates multi-SLO serving as a constrained optimization problem and introduces a hardware-aware algorithm.<n>It features a speculate-select-verify pipeline that enables fine-grained control over decoding speed while maximizing system throughput.
  arXiv  Detail & Related papers  (2025-01-21T14:15:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.