ExFold: Unified Expert Folding for Training-Free MoE Prefill-Decode Acceleration

• URL: http://arxiv.org/abs/2608.24938v1
• Date: Mon, 24 Aug 2026 08:53:58 GMT
• Title: ExFold: Unified Expert Folding for Training-Free MoE Prefill-Decode Acceleration
• Abstract summary: Mixture-of-Experts (MoE) models scale capacity for strong quality while keeping per-token compute through expert activation.<n>We propose ExFold, jointly accelerating MoE prefill decode decode and batch-level expert-pool folding.
• Score: 28.21219412744595
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Mixture-of-Experts (MoE) models scale capacity for strong quality while keeping per-token compute bounded through sparse expert activation. Yet low-latency MoE serving is increasingly challenging, because it spans two inference phases with fundamentally different bottlenecks: prefill is dominated by token-wise expert computation, whereas decode is constrained by memory traffic from the batch-wise activated expert set. However, existing training-free acceleration methods optimize only a single resource proxy, either the experts each token executes or the experts a batch activates, and either discard the excluded experts' contribution or leave it only implicitly approximated. In this paper, we propose ExFold, a unified training-free expert-folding framework for jointly accelerating MoE prefill and decode. ExFold casts both prefill and decode as one budgeted output-approximation problem: execute only a phase-specific constrained expert set while projecting the contribution of budget-excluded experts onto retained experts using calibrated scalar projectors. Motivated by the observation that many expert outputs are directionally aligned but differ in magnitude, ExFold calibrates a pairwise scalar-projector matrix on unlabeled data and uses it at inference time to fold excluded expert contributions into retained experts. Under this view, prefill acceleration becomes token-level Top-K folding, and decode acceleration becomes batch-level expert-pool folding. The two phases differ only in how retained experts are selected, while excluded contributions are recovered by one shared folding mechanism. We implement ExFold as a plug-and-play plugin in vLLM, with a lightweight expert-folding CUDA kernel, delivering up to 1.41x TTFT and 2.45x TPOT speedups while retaining about 99% of the original average quality.

Related papers

• APEX: Adaptive Expert Prefetching for Memory-Efficient Edge MoE Inference [0.0]
  Mixture-of-Experts (MoE) models are attractive for edge deployment because they provide high model capacity while activating only a small subset of parameters per token.<n>We present APEX: Adaptive Expert Prefetching, a predictive resource management framework that overlaps expert loading with useful computation.
  arXiv  Detail & Related papers  (2026-08-12T05:57:09Z)
• Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation [49.8459545291965]
  Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.<n>We study this dependency by decomposing sparsely upcycled feed-forward experts into key-value channels.<n>We instantiate it in UniF-MoE, a unified framework for token-adaptive MoE computation.
  arXiv  Detail & Related papers  (2026-08-11T02:40:58Z)
• Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts [34.20364644493512]
  We propose a cost-aware speculative decoding framework that incorporates predicted marginal expert activation cost into draft selection.<n>We evaluate textscEcoSpec on three large-scale MoE models, including DeepSeek-V3.1 (671B), Qwen3-235B-A22B, and GPT-OSS-120B.
  arXiv  Detail & Related papers  (2026-07-14T12:22:55Z)
• SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning [83.66308307152808]
  We propose StAbilized Mixture-of-Experts (SAME) for Multimodal Continual Instruction Tuning (MCIT)<n>SAME stabilizes expert selection by decomposing routing dynamics into subspaces and updating only task-relevant directions.<n>It also introduces adaptive expert activation to freeze selected experts during training, reducing redundant and cross-task interference.
  arXiv  Detail & Related papers  (2026-02-02T11:47:06Z)
• $\infty$-MoE: Generalizing Mixture of Experts to Infinite Experts [43.075289015406355]
  Mixture of Experts (MoE) selects a few feed-forward networks (FFNs) per token, achieving an effective trade-off between computational cost and performance.<n>We propose $infty$-MoE that selects a portion of the parameters of large FFNs based on continuous values sampled for each token.<n>Experiments show that a GPT-2 Small-based $infty$-MoE model, with 129M active and 186M total parameters, achieves comparable performance to a dense GPT-2 Medium with 350M parameters.
  arXiv  Detail & Related papers  (2026-01-25T03:55:51Z)
• Beyond Redundancy: Diverse and Specialized Multi-Expert Sparse Autoencoder [59.89996751196727]
  Sparse autoencoders (SAEs) have emerged as a powerful tool for interpreting large language models.<n>SAEs' hidden layers have high dimensionality to satisfy sparsity constraints, resulting in prohibitive training and inference costs.<n>Recent Mixture of Experts (MoE) approaches attempt to address this by SAEs into narrower expert networks with gated activation.<n>We propose two key innovations: (1) Multiple Expert Activation that simultaneously engages semantically weighted expert subsets to encourage specialization, and (2) Feature Scaling that enhances diversity through adaptive high-frequency scaling.
  arXiv  Detail & Related papers  (2025-11-07T22:19:34Z)
• Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging [17.490596264046435]
  Sub-MoE is a novel MoE compression framework via Subspace Expert Merging.<n>Our key insight is to perform joint Singular Value Decomposition (SVD) ond expert weights.<n>Our Sub-MoE significantly outperforms existing expert pruning and merging methods.
  arXiv  Detail & Related papers  (2025-06-29T14:43:50Z)
• Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations [48.890534958441016]
  This study investigates domain specialization and expert redundancy in large-scale MoE models.<n>We propose a simple yet effective pruning framework, EASY-EP, to identify and retain only the most relevant experts.<n>Experiments on DeepSeek-R1 and DeepSeek-V3-0324 show that our method can achieve comparable performances and $2.99times$ throughput under the same memory budget with full model with only half the experts.
  arXiv  Detail & Related papers  (2025-04-09T11:34:06Z)
• HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference [54.40808356999408]
  We present HOBBIT, a mixed precision expert offloading system to enable flexible and efficient MoE inference. Our key insight is that dynamically replacing less critical cache-miss experts with low precision versions can substantially reduce expert-loading latency. HOBBIT achieves up to a 9.93x speedup in decoding compared to state-of-the-art MoE offloading systems.
  arXiv  Detail & Related papers  (2024-11-03T04:25:46Z)
• MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts [63.67734699877724]
  MoE++ is a general and heterogeneous MoE framework that integrates both Feed-Forward Network(FFN) and zero-computation experts. MoE++ achieves better performance while delivering 1.1-2.1x expert forward throughput compared to a vanilla MoE model of the same size.
  arXiv  Detail & Related papers  (2024-10-09T18:01:27Z)
• Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts [44.09546603624385]
  We introduce a notion of expert specialization for Soft MoE. We show that when there are many small experts, the architecture is implicitly biased in a fashion that allows us to efficiently approximate the specialized expert subset.
  arXiv  Detail & Related papers  (2024-09-02T00:39:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.