ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform

• URL: http://arxiv.org/abs/2607.26566v1
• Date: Wed, 29 Jul 2026 07:39:56 GMT
• Title: ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform
• Abstract summary: Text-to-image (T2I) are increasingly deployed on serverless platforms because users often compose customized and invoke them intermittently.<n>Existing platforms typically deploy each workflow as an opaque GPU function, provisioning, placing, and scaling all constituent models in the workflow together.<n>We present ServerlessT2I, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled.
• Score: 12.291915972120902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-image (T2I) workflows are increasingly deployed on serverless platforms because users often compose customized workflows and invoke them intermittently. Existing platforms typically deploy each workflow as an opaque GPU function, provisioning, placing, and scaling all constituent models in the workflow together. This monolithic design obscures workflow structure, inflates scaling overhead, forces users to manage low-level GPU coordination, and limits fine-grained fairness in multi-tenant clusters. In this paper, we present ServerlessT2I, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled. By explicitly managing individual model execution, ServerlessT2I enables per-model scaling, declarative workflow composition, transparent GPU-resident communication, and fairness-aware scheduling. To make this decomposition efficient, ServerlessT2I harvests slack GPU memory left idle by compute-bound T2I inference to build a data plane that reduces model loading and data communication overheads. \sys{} further introduces a fair scheduler for multi-tenant serving. Using production traces, ServerlessT2I sustains up to 2$\times$ higher request rates than existing T2I workflow serving systems with the same GPU budget; for a fixed request rate, it saves up to 3$\times$ GPU resources while satisfying service level objectives (SLOs).

Related papers

• GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Serving [27.270604867405325]
  GraphFlow is a system that efficiently integrates into agent serving through two key designs.<n>It consistently outperforms state-of-the-art methods, yielding an average performance improvement of approximately 4.95 percentage points.
  arXiv  Detail & Related papers  (2026-05-21T14:45:40Z)
• RollArt: Scaling Agentic RL Training via Disaggregated Infrastructure [49.88201789074532]
  Agentic Reinforcement Learning (RL) enables Large Language Models (LLMs) to perform autonomous decision-making and long-term planning.<n>We present RollArc, a distributed system designed to maximize throughput for multi-task agentic RL on disaggregated infrastructure.
  arXiv  Detail & Related papers  (2025-12-27T11:14:23Z)
• Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms [7.6151320996822625]
  Murakkab is a resource-efficient serving system for agentic AI applications.<n>We show that Murakkab reduces usage by up to 2.8$times$, energy consumption by 3.7$times$, and cost by 4.3$times$ while maintaining service-level objectives (SLOs)<n>Our evaluation on diverse GPU shows that Murakkab reduces GPU usage by up to 2.8$times$, energy consumption by 3.7$times$, and cost by 4.3$times$ while maintaining SLOs.
  arXiv  Detail & Related papers  (2025-08-22T17:41:27Z)
• WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models [105.46456444315693]
  We presentLLM, a data-centric framework to enhance the capability of large language models in workflow orchestration. It first constructs a large-scale fine-tuningBench with 106,763 samples, covering 1,503 APIs from 83 applications across 28 categories. LlamaLlama demonstrates a strong capacity to orchestrate complex APIs, while also achieving notable generalization performance.
  arXiv  Detail & Related papers  (2024-11-08T09:58:02Z)
• FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression [55.992528247880685]
  Decentralized training faces significant challenges regarding system design and efficiency. We present FusionLLM, a decentralized training system designed and implemented for training large deep neural networks (DNNs) We show that our system and method can achieve 1.45 - 9.39x speedup compared to baseline methods while ensuring convergence.
  arXiv  Detail & Related papers  (2024-10-16T16:13:19Z)
• Benchmarking Agentic Workflow Generation [80.74757493266057]
  We introduce WorfBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures.<n>We also present WorfEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms.<n>We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
  arXiv  Detail & Related papers  (2024-10-10T12:41:19Z)
• SeBS-Flow: Benchmarking Serverless Cloud Function Workflows [51.4200085836966]
  We propose the first serverless workflow benchmarking suite SeBS-Flow.<n>SeBS-Flow includes six real-world application benchmarks and four microbenchmarks representing different computational patterns.<n>We conduct comprehensive evaluations on three major cloud platforms, assessing performance, cost, scalability, and runtime deviations.
  arXiv  Detail & Related papers  (2024-10-04T14:52:18Z)
• SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules [9.172732822379862]
  Text-to-image (T2I) generation using diffusion models has become a blockbuster service in today's AI cloud.<n>A production T2I service typically involves a serving workflow where a base diffusion model is augmented with various "add-on" modules.<n>We present SwiftDiffusion, a system that efficiently serves a T2I workflow through a holistic approach.
  arXiv  Detail & Related papers  (2024-07-02T07:59:08Z)
• Compass: A Decentralized Scheduler for Latency-Sensitive ML Workflows [0.792324422300924]
  We consider ML query processing in distributed systems where GPU-enabled workers coordinate to execute complex queries. In such systems, coscheduling of GPU memory management and task placement represents a promising opportunity. We propose Compass, a novel framework that unifies these functions to reduce job latency while using resources efficiently.
  arXiv  Detail & Related papers  (2024-02-27T16:21:28Z)
• M$^3$ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design [95.41238363769892]
  Multi-task learning (MTL) encapsulates multiple learned tasks in a single model and often lets those tasks learn better jointly. Current MTL regimes have to activate nearly the entire model even to just execute a single task. We present a model-accelerator co-design framework to enable efficient on-device MTL.
  arXiv  Detail & Related papers  (2022-10-26T15:40:24Z)
• Taskflow: A Lightweight Parallel and Heterogeneous Task Graph Computing System [12.813275501138193]
  Taskflow aims to streamline the building of parallel and heterogeneous applications using a lightweight task graph-based approach. Our programming model distinguishes itself as a very general class of task graph parallelism with in-graph control flow. We have demonstrated the promising performance of Taskflow in real-world applications.
  arXiv  Detail & Related papers  (2020-04-23T00:21:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.