AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding

• URL: http://arxiv.org/abs/2608.02989v1
• Date: Tue, 04 Aug 2026 01:01:12 GMT
• Title: AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding
• Abstract summary: Speculative decoding verifies a tree of draft tokens in one target-model forward pass.<n>For a mixture-of-experts (MoE) target, parallel verification can activate the union of the experts selected by all tree nodes.<n>We introduce AcceptMoE, a verifier-side expert selector that automatically adjusts the number of eligible experts for each verification block.
• Score: 21.76650680783374
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speculative decoding verifies a tree of draft tokens in one target-model forward pass. For a mixture-of-experts (MoE) target, however, parallel verification can activate the union of the experts selected by all tree nodes, even though only a small subset of those nodes reaches the accepted output. Token count, activated-expert union size, and expert-weight traffic are therefore distinct cost measures: reducing the token workload need not shrink the expert union proportionally, and under offloading, transfer traffic also depends on cache residency. We introduce AcceptMoE, a verifier-side expert selector that combines target-router scores with offline-estimated commitment probabilities and automatically adjusts the number of eligible experts for each verification block, eliminating the need for a user-specified expert budget. Under offloading, AcceptMoE conditions expert eligibility on cache residency instead of predicting natural routes and prefetching the corresponding expert weights. Although constraining target-expert eligibility changes the model distribution, across 12 model-task pairs spanning three MoE targets and four benchmarks, AcceptMoE's mean accuracy is 0.27 percentage points lower than that of EAGLE-3 speculative decoding with natural routing. Served with SGLang at batch size one, it reaches 1.290 times the throughput of this baseline with all expert weights in GPU memory, and 2.06 times under physical expert offloading, while reducing host-to-device traffic by 73.6 percent to 77.1 percent.

Related papers

• ACE: Adaptive Calibration-Free Expert Skipping for MoE-based LLMs [15.24786135265898]
  Mixture-of-Experts (MoE) architectures provide an efficient paradigm for scaling large language models (LLMs)<n>We propose ACE, a training-free, calibration-free, and checkpoint-preserving framework for token-adaptive expert skipping.
  arXiv  Detail & Related papers  (2026-09-04T14:56:00Z)
• Verification-Aware Training for Speculative Decoding [57.84976863792784]
  We introduce Verification-Aware Training (VAT), a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision.<n>VAT modifies only the training objective, so it can be layered on top of existing methods without changing the draft architecture, the target model, or the inference procedure.<n>VAT improves average acceptance length by up to 11.4% and wall-clock speedup by up to 8.7%, with consistent gains across math, code, and chat benchmarks.
  arXiv  Detail & Related papers  (2026-08-31T01:42:10Z)
• Meta-Learning Where to Allocate Experts: Task-Conditioned Layer-Wise Compression for MoEs [8.622261628194893]
  Mixture-of-Experts (MoE) models route each token to a subset of expert networks, increasing capacity while keeping per-token sparse.<n>We propose MetaNet, a support-set controller that predicts, for each layer, an expert-retention threshold and a bounded routing bias.
  arXiv  Detail & Related papers  (2026-08-27T06:01:46Z)
• Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation [49.8459545291965]
  Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.<n>We study this dependency by decomposing sparsely upcycled feed-forward experts into key-value channels.<n>We instantiate it in UniF-MoE, a unified framework for token-adaptive MoE computation.
  arXiv  Detail & Related papers  (2026-08-11T02:40:58Z)
• DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference [0.4944564023471818]
  DraftExpert is an expansion-aware self-speculative decoding framework for expert-offloaded MoE inference.<n>On DeepSeek-V2-Lite and Moonlight-16B-A3B across CPU-GPU and Flash-NPU offload, DraftExpert improves decode throughput by 1.45x on average.
  arXiv  Detail & Related papers  (2026-07-27T13:44:42Z)
• Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts [34.20364644493512]
  We propose a cost-aware speculative decoding framework that incorporates predicted marginal expert activation cost into draft selection.<n>We evaluate textscEcoSpec on three large-scale MoE models, including DeepSeek-V3.1 (671B), Qwen3-235B-A22B, and GPT-OSS-120B.
  arXiv  Detail & Related papers  (2026-07-14T12:22:55Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• dMoE: dLLMs with Learnable Block Experts [71.572316901001]
  We propose dMoE, a simple yet effective block-level MoE framework.<n>We show that dMoE substantially reduces the number of uniquely activated experts during inference.<n>It also reduces memory usage by 76.64% to 79.84% and achieves 1.14$times$ to 1.66$times$ end-to-end latency speedup.
  arXiv  Detail & Related papers  (2026-05-29T06:03:50Z)
• Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding [15.040000425985324]
  Tree-based speculative decoding accelerates autoregressive generation by verifying multiple draft candidates in parallel, but this advantage weakens for sparse Mixture-of-Experts (MoE) models.<n>We propose EVICT, a training-free, hyper parameter-free, and lossless adaptive verification method for MoE speculative decoding.<n> EVICT makes every verified token count by truncating the draft tree before target verification and retaining only the cost-effective prefix.
  arXiv  Detail & Related papers  (2026-05-01T01:52:01Z)
• ERMoE: Eigen-Reparameterized Mixture-of-Experts for Stable Routing and Interpretable Specialization [13.182475975397251]
  ERMoE is a sparse MoE transformer that replaces learned gating logits with an "Eigenbasis Score"<n>We show that ERMoE achieves state-of-the-art accuracy on ImageNet classification and cross-modal image-text retrieval benchmarks.<n>A 3D MRI variant (ERMoE-ba) improves brain age prediction accuracy by more than 7% and yields interpretable expert specializations.
  arXiv  Detail & Related papers  (2025-11-14T05:31:37Z)
• MC#: Mixture Compressor for Mixture-of-Experts Large Models [86.64315380917827]
  Mixture-of-Experts (MoE) effectively scales large language models (LLMs) and vision-language models (VLMs) by increasing capacity through sparse activation.<n>We propose MC# (Mixture-Compressor-sharp), a framework that combines static quantization and dynamic expert pruning.
  arXiv  Detail & Related papers  (2025-10-13T03:12:46Z)
• Ban&Pick: Ehancing Performance and Efficiency of MoE-LLMs via Smarter Routing [36.625445642399136]
  Ban&Pick is a post-training, plug-and-play strategy for smarter routing.<n>It discovers and reinforces key experts with outsized impact on performance.<n>It delivers free performance gains and inference acceleration without retraining or architectural changes.
  arXiv  Detail & Related papers  (2025-09-08T05:38:10Z)
• HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference [54.40808356999408]
  We present HOBBIT, a mixed precision expert offloading system to enable flexible and efficient MoE inference. Our key insight is that dynamically replacing less critical cache-miss experts with low precision versions can substantially reduce expert-loading latency. HOBBIT achieves up to a 9.93x speedup in decoding compared to state-of-the-art MoE offloading systems.
  arXiv  Detail & Related papers  (2024-11-03T04:25:46Z)
• MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts [63.67734699877724]
  MoE++ is a general and heterogeneous MoE framework that integrates both Feed-Forward Network(FFN) and zero-computation experts. MoE++ achieves better performance while delivering 1.1-2.1x expert forward throughput compared to a vanilla MoE model of the same size.
  arXiv  Detail & Related papers  (2024-10-09T18:01:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.