GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving

• URL: http://arxiv.org/abs/2606.13501v1
• Date: Thu, 11 Jun 2026 15:51:10 GMT
• Title: GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving
• Abstract summary: GF-DiT is a policy-programmable runtime for elastic Diffusion Transformers (DiTs)<n>It adapts the parallelism of running requests according to workload demands and service objectives.<n>It improves throughput by up to 6.01$times$, reduces mean latency by up to 95%, SLO violation rates by up to 90%, and reduces communication-group setup overhead from 778 ms to approximately 60 $$s.
• Score: 36.55701690878634
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion Transformers (DiTs) have become the dominant architecture for image and video generation, creating growing demand for efficient DiT serving. Existing systems assign each request a fixed parallel configuration throughout its lifetime. However, DiT workloads exhibit substantial heterogeneity across requests, execution stages, and system conditions, making static parallelism inefficient and often leading to poor GPU utilization and degraded service quality. This paper argues that DiT serving should treat GPU parallelism as a first-class schedulable resource. We present GF-DiT, a policy-programmable runtime for elastic DiT serving that dynamically adapts the parallelism of running requests according to workload demands and service objectives. GF-DiT introduces an asynchronous execution abstraction that decomposes requests into independently schedulable trajectory tasks and enables online GPU reallocation. To make elastic parallelism practical, GF-DiT further proposes group-free collectives, a lightweight communication abstraction that supports low-overhead online formation and reconfiguration of arbitrary execution groups. We implement GF-DiT in vLLM-Omni and evaluate it on representative image and video diffusion workloads. Compared with fixed-pipeline execution with static parallelism, GF-DiT improves throughput by up to 6.01$\times$, reduces mean latency by up to 95%, lowers SLO violation rates by up to 90%, and reduces communication-group setup overhead from 778 ms to approximately 60 $μ$s.

Related papers

• DiscreteRTC: Discrete Diffusion Policies are Natural Asynchronous Executors [57.944744187489185]
  We propose DiscreteRTC, which replaces external corrections with native unmasking.<n>DiscreteRTC is simpler to implement with 0 lines of code for async inpainting, faster at inference with only 0.7x computation compared with generating actions from scratch, and better at execution with 50% higher success rate in real-world dynamic pick task compared with flow-based RTC.
  arXiv  Detail & Related papers  (2026-04-27T23:04:03Z)
• Hybrid JIT-CUDA Graph Optimization for Low-Latency Large Language Model Inference [0.3580891736370874]
  This paper presents a hybrid framework that combines Just-In-Time (JIT) compilation with runtime Graph execution to reduce launch overhead.<n>We evaluate the proposed approach on LLaMA-2 7B using single-GPU, batch-size-one inference across prompt lengths from 10 to 500 tokens.
  arXiv  Detail & Related papers  (2026-04-25T23:19:53Z)
• ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism [0.0]
  Large-scale decentralized training can utilize otherwise untapped compute resources.<n>Recent efforts have claimed up to 100x activation compression but rely on complex constrained optimization and diverge from true end-to-end training.<n>We show that ResBMs achieve state-of-the-art 128x activation compression without significant loss in convergence rates and without significant memory or compute overhead.
  arXiv  Detail & Related papers  (2026-04-13T18:40:45Z)
• Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? [48.59679063480356]
  Diffusion Language Models (DLMs) are often advertised as enabling parallel token generation, yet practical fast DLMs converge to left-to-right, autoregressive (AR)-like decoding dynamics.<n>We argue that a primary driver of AR-like decoding is a mismatch between DLM objectives and the highly sequential structure of widely used training data.<n>Motivated by this diagnosis, we propose NAP (Non-Autoregressive Parallel DLMs), a proof-of-concept, data-centric approach that better aligns supervision with non-AR parallel decoding.
  arXiv  Detail & Related papers  (2026-02-26T17:04:57Z)
• Canzona: A Unified, Asynchronous, and Load-Balanced Framework for Distributed Matrix-based Optimizers [36.650880799066215]
  Asynchronous approaches suffer from computational redundancy, while layer-wise partitioning fails to reconcile this conflict.<n>For Data Parallelism, we introduce an alpha-Balanced Static Partitioning strategy that respects atomicity while neutralizing the load imbalance.<n>Our approach achieves a 1.57x speedup in end-to-end time and reducing step latency by 5.8x compared to the baseline.
  arXiv  Detail & Related papers  (2026-02-04T07:38:24Z)
• CSGO: Generalized Optimization for Cold Start in Wireless Collaborative Edge LLM Systems [62.24576366776727]
  We propose a latency-aware scheduling framework to minimize total inference latency.<n>We show that the proposed method significantly reduces cold-start latency compared to baseline strategies.
  arXiv  Detail & Related papers  (2025-08-15T07:49:22Z)
• HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving [10.130938079844121]
  HedraRAG is a runtime system built on a graph-based abstraction that exposes optimization opportunities across stage-level parallelism, intra-request similarity, and inter-request skewness.<n>The resulting execution plans are mapped onto hybrid CPU-GPU pipelines to improve resource utilization and reduce latency.
  arXiv  Detail & Related papers  (2025-07-12T04:42:43Z)
• Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism [59.79227116582264]
  Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging.<n>We propose a novel compression algorithm that compresses both forward and backward passes, enabling up to 99% compression with no convergence degradation.
  arXiv  Detail & Related papers  (2025-06-02T02:19:22Z)
• xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism [5.704297874096985]
  Diffusion models are pivotal for generating high-quality images and videos. This paper introduces xDiT, a comprehensive parallel inference engine for DiTs. Notably, we are the first to demonstrate DiTs scalability on Ethernet-connected GPU clusters.
  arXiv  Detail & Related papers  (2024-11-04T01:40:38Z)
• FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression [55.992528247880685]
  Decentralized training faces significant challenges regarding system design and efficiency. We present FusionLLM, a decentralized training system designed and implemented for training large deep neural networks (DNNs) We show that our system and method can achieve 1.45 - 9.39x speedup compared to baseline methods while ensuring convergence.
  arXiv  Detail & Related papers  (2024-10-16T16:13:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.