TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offload

• URL: http://arxiv.org/abs/2605.20179v1
• Date: Tue, 19 May 2026 17:59:08 GMT
• Title: TIDE: Efficient and Lossless MoE Diffusion LLM Inference with I/O-aware Expert Offload
• Abstract summary: Diffusion Large Language Models (dLLMs) offer better hardware utilization and bidirectional context through parallel block-level decoding.<n>Existing AR-based methods often incur either prohibitive I/O overhead or significant compute bottlenecks.<n>We propose TIDE, a novel resource-efficient inference system that leverages the temporal stability of expert activations.
• Score: 28.278474158271894
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive (AR) models, offering better hardware utilization and bidirectional context through parallel block-level decoding. However, as dLLMs continue to scale up with mixture-of-experts (MoE) architectures, their deployment on resource-constrained devices remains an open challenge. Existing AR-based methods often incur either prohibitive I/O overhead or significant compute bottlenecks. In this work, we propose TIDE, a novel resource-efficient inference system that leverages the temporal stability of expert activations during the diffusion process within the block. Specifically, we leverage the temporal stability of expert activations during the diffusion process within the block and introduce an interval-based expert refresh strategy that updates the expert placement in an I/O-aware fashion. To ensure optimal performance, we formulate the inference scheduling as a mathematical programming problem, solving for the optimal interval that minimizes I/O traffic and CPU computation. Most importantly, TIDE is a lossless optimization that requires no model training, providing a "free lunch" acceleration for dLLM inference. In a single GPU-CPU system, we demonstrate that TIDE achieves up to 1.4$\times$ and 1.5$\times$ throughput improvements over prior baselines on LLaDA2.0-mini and LLaDA2.0-flash models, respectively.

Related papers

• Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice [12.809268527016599]
  We introduce geometry-aware online scheduling by introducing Smallest Volume First (SVF) algorithm and its highly efficient variant, 1-bit SVF.<n>SVF delivers strong reductions in both average and tail latency, while 1-bit SVF, with merely a single bit information, achieves competitive throughput and latency.<n>This work establishes a theoretically sound and empirically proven approach for resolving memory-constrained scheduling in modern Large Language Model deployments.
  arXiv  Detail & Related papers  (2026-06-21T04:05:38Z)
• FLARE: Diffusion for Hybrid Language Model [72.60770374799634]
  FLARE is a systematic conversion framework for hybrid-attention large language models.<n>It combines a token-equal AR-and-diffusion objective, hardware-aware kernels, and unified inference, enabling one checkpoint to support both AR-style verified decoding and diffusion-style parallel denoising.<n>Our results suggest practical dLLMs are limited not only by decoding algorithms, but also by transfer data quality and the training inefficiency of current block-diffusion objectives.
  arXiv  Detail & Related papers  (2026-06-01T06:58:15Z)
• Flux Attention: Context-Aware Hybrid Attention for Efficient LLMs Inference [36.81958522560282]
  We introduce Flux Attention, a context-aware framework that dynamically optimize attention at the layer level.<n>As a parameter-efficient approach, our framework requires only 12 hours of training on 8$times$A800 GPUs.<n>Our framework achieves a superior trade-off between performance and inference speed compared with baseline models, with speed improvements of up to $2.8times$ and $2.0times$ in the prefill and decode stages.
  arXiv  Detail & Related papers  (2026-04-08T07:36:17Z)
• DiRL: An Efficient Post-Training Framework for Diffusion Language Models [54.405206032785706]
  Diffusion Language Models (dLLMs) have emerged as promising alternatives to Auto-Regressive (AR) models.<n>Existing methods suffer from computational inefficiency and objective mismatches between training and inference.<n>We introduce DiRL, an efficient post-training framework that tightly integrates FlexAttention-accelerated blockwise training with LMDeploy-optimized inference.
  arXiv  Detail & Related papers  (2025-12-23T08:33:19Z)
• xLLM Technical Report [57.13120905321185]
  We introduce xLLM, an intelligent and efficient Large Language Model (LLM) inference framework.<n>xLLM builds a novel decoupled service-engine architecture.<n>xLLM-Engine co-optimizes system and algorithm designs to fully saturate computing resources.
  arXiv  Detail & Related papers  (2025-10-16T13:53:47Z)
• dInfer: An Efficient Inference Framework for Diffusion Language Models [54.80918957287927]
  Diffusion-based large language models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs.<n>We present dInfer, an efficient and framework for dLLM inference.
  arXiv  Detail & Related papers  (2025-10-09T16:19:42Z)
• Fast-dLLM v2: Efficient Block-Diffusion LLM [64.38006546510337]
  Fast-dLLM v2 is a block diffusion language model that adapts pretrained AR models into dLLMs for parallel text generation.<n>This represents a 500x reduction in training data compared to full-attention diffusion LLMs such as Dream (580B tokens)
  arXiv  Detail & Related papers  (2025-09-30T14:40:18Z)
• Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition [95.54406667705999]
  Pangu Embedded is an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs)<n>It addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs.<n>It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture.
  arXiv  Detail & Related papers  (2025-05-28T14:03:02Z)
• Online Scheduling for LLM Inference with KV Cache Constraints [22.133592174540052]
  Large Language Model (LLM) inference is an intensive process requiring efficient scheduling to optimize latency and resource utilization.<n>We propose a novel and theoretically scheduling algorithm that minimizes inference latency while effectively managing the KV cache's memory.
  arXiv  Detail & Related papers  (2025-02-10T23:11:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.