FMplex: Model Virtualization for Serving Extensible Foundation Models

• URL: http://arxiv.org/abs/2606.09643v1
• Date: Mon, 08 Jun 2026 15:38:16 GMT
• Title: FMplex: Model Virtualization for Serving Extensible Foundation Models
• Abstract summary: Existing modelserving systems deploy each customized task as an independent model.<n> FMplex lets independently customized tasks share a backbone while preserving task-specific extensions.<n>Across 7 FM backbones and 92 downstream tasks, FMplex reduces latency to 80% over best-effort co-location.
• Score: 2.5151309501683454
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation models (FMs) are increasingly used as backbones for downstream tasks across language, vision, time-series, and multimodal applications. Yet existing model-serving systems deploy each customized task as an independent model instance, thereby replicating heavyweight backbones, wasting accelerator memory, and losing opportunities to amortize batching and loading costs. This paper presents FMplex, a serving system that treats FM backbones as a virtualization substrate for deployment sharing. FMplex presents each task with a virtual foundation model (vFM), a logically private FM instance backed by a shared physical FM. This abstraction lets independently customized tasks share a backbone while preserving task-specific extensions, independent lifecycles, and task-level isolation. In addition, we propose a batch-aware fair-queueing scheduler that combines weighted task-level sharing with inter- and intra-task batching across colocated tasks. We implement a FMplex-based serving stack spanning task construction, sharing-aware deployment, and runtime execution. Across 7 FM backbones (16 variants) and 92 downstream tasks, FMplex reduces latency by up to 80% over spatial partitioning and 33.3% over best-effort co-location, while hosting up to 6x more tasks at cluster scale.

Related papers

• Task-Blind No MORE: Multi-Task Information Flow in Unified Ranking Backbones [15.647515309133025]
  MORE (Multi-task cO-evolving Ranking modEl) embeds multi-task information flow inside the backbone.<n>It enables task-specific signals to co-evolve with sequence and feature representations at every layer rather than in a post-hoc fusion.<n> MORE consistently outperforms baselines across all tasks under comparable parameter and FLOPs budgets, and scales well with model size.
  arXiv  Detail & Related papers  (2026-09-07T09:36:57Z)
• FMTK: A Modular Toolkit for Composable Time Series Foundation Model Pipelines [2.6650725474904617]
  Time-series foundation models (TSFMs) have shown strong performance on classification, regression, and imputation tasks.<n>Recent pipelines combine TSFMs with task-specific encoders, decoders, and adapters to improve performance.<n>We introduce FMTK, an open-source, lightweight and implementations toolkit for constructing and fine-tuning TSFM pipelines.
  arXiv  Detail & Related papers  (2025-11-30T19:14:04Z)
• StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasets [14.867396697566257]
  We extend the partial learning setup to a zero-shot setting, training a multi-task model on multiple datasets, each labeled for only a subset of tasks.<n>Our method, StableMTL, repurposes image generators for latent regression.<n>Instead of per-task losses requiring careful balancing, a unified latent loss is adopted, enabling seamless scaling to more tasks.
  arXiv  Detail & Related papers  (2025-06-09T17:59:59Z)
• Intelligent Orchestration of Distributed Large Foundation Model Inference at the Edge [46.1232919707345]
  Large Foundation Models (LFMs) promise to unlock new capabilities for next-generation Edge AI applications.<n>Current split inference strategies, which partition LFM layers across nodes, are not designed to adapt to fluctuating workloads.<n>We propose a novel adaptive split inference orchestration framework that elevates both the placement and partitioning of LFM layers to runtime-tunable variables.
  arXiv  Detail & Related papers  (2025-03-19T15:35:56Z)
• OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models [72.8156832931841]
  Generalist models are capable of performing diverse multi-modal tasks in a task-agnostic way within a single model. We release a generalist model learning system, OFASys, built on top of a declarative task interface named multi-modal instruction.
  arXiv  Detail & Related papers  (2022-12-08T17:07:09Z)
• M$^3$ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design [95.41238363769892]
  Multi-task learning (MTL) encapsulates multiple learned tasks in a single model and often lets those tasks learn better jointly. Current MTL regimes have to activate nearly the entire model even to just execute a single task. We present a model-accelerator co-design framework to enable efficient on-device MTL.
  arXiv  Detail & Related papers  (2022-10-26T15:40:24Z)
• Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners [67.5865966762559]
  We study whether sparsely activated Mixture-of-Experts (MoE) improve multi-task learning. We devise task-aware gating functions to route examples from different tasks to specialized experts. This results in a sparsely activated multi-task model with a large number of parameters, but with the same computational cost as that of a dense model.
  arXiv  Detail & Related papers  (2022-04-16T00:56:12Z)
• Controllable Dynamic Multi-Task Architectures [92.74372912009127]
  We propose a controllable multi-task network that dynamically adjusts its architecture and weights to match the desired task preference as well as the resource constraints. We propose a disentangled training of two hypernetworks, by exploiting task affinity and a novel branching regularized loss, to take input preferences and accordingly predict tree-structured models with adapted weights.
  arXiv  Detail & Related papers  (2022-03-28T17:56:40Z)
• MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning [61.28547338576706]
  Population-based multi-agent reinforcement learning (PB-MARL) refers to the series of methods nested with reinforcement learning (RL) algorithms. We present MALib, a scalable and efficient computing framework for PB-MARL.
  arXiv  Detail & Related papers  (2021-06-05T03:27:08Z)
• Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields [67.51177964010967]
  We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks. We find that explicit modeling of inter-dependencies between task predictions outperforms single-task as well as standard multi-task models.
  arXiv  Detail & Related papers  (2020-05-01T07:11:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.