DBES: A Systematic Benchmark and Metric Suite for Evaluating Expert Specialization in Large-Scale MoEs

• URL: http://arxiv.org/abs/2605.18498v1
• Date: Mon, 18 May 2026 14:50:30 GMT
• Title: DBES: A Systematic Benchmark and Metric Suite for Evaluating Expert Specialization in Large-Scale MoEs
• Abstract summary: Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood.<n>We introduce DBES, a comprehensive diagnostic framework combining a multi-domain benchmark with five theoretically grounded metrics.<n>This work provides the first systematic methodology for evaluating expert specialization independently of accuracy metrics.
• Score: 4.089532981955943
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood, with traditional evaluations conflating architectural load-balancing with functional specialization. We introduce DBES, a comprehensive diagnostic framework combining a multi-domain benchmark with five theoretically grounded metrics: Routing Specialization, Normalized Effective Rank, Domain Isolation, Routing Stiffness Score, and N-gram Expertise measures. Critical findings demonstrate distinct specialization paradigms across models: Qwen-series exhibit modular specialization with high domain isolation, while DeepSeek and GLM employ distributed collaboration. However, we emphasize that specialization is a diagnostic dimension, necessary but not sufficient for downstream performance. Most crucially, interventional evidence validates the actionability of these metrics: by using DBES to identify high-specialization expert paths during domain-specific post-training, we achieved 66% to 94.48% improvement in specialized domains with only 15% of original training resources, demonstrating that these diagnostic tools can be converted into concrete optimization operators. This work provides the first systematic methodology for evaluating expert specialization independently of accuracy metrics, offering crucial insights for the design and post-training optimization of next-generation MoE systems.

Related papers

• SMoES: Soft Modality-Guided Expert Specialization in MoE-VLMs [9.733867547637956]
  We propose Soft Modality-guided Expert (SMoES) for large vision models (VLMs)<n>SMoES consists of dynamic soft modality scores that capture layer-dependent fusion patterns, an expert binning mechanism aligned with expert-parallel deployment, and an inter-bin mutual information regularization.<n> Experiments across four MoE-basedVLMs and 16 benchmarks demonstrate improvement on both effectiveness and efficiency.
  arXiv  Detail & Related papers  (2026-04-27T03:23:19Z)
• DeferredSeg: A Multi-Expert Deferral Framework for Trustworthy Medical Image Segmentation [66.24948946364329]
  We introduce DeferredSeg, a deferral-aware segmentation framework that defers predictions to human experts in specific regions.<n>We evaluate DeferredSeg on three challenging medical datasets using MedSAM and CENet as the base segmentor for fair comparison.<n> Experimental results show that DeferredSeg consistently outperforms the baseline.
  arXiv  Detail & Related papers  (2026-04-14T07:50:31Z)
• MoE Lens -- An Expert Is All You Need [0.09444932939944471]
  Mixture of Experts (MoE) models enable parameter-efficient scaling through sparse expert activations.<n>We present a systematic analysis of expert specialization in MoEs through two complementary approaches.
  arXiv  Detail & Related papers  (2026-03-06T01:35:28Z)
• SD-MoE: Spectral Decomposition for Effective Expert Specialization [29.649486549025138]
  Mixture-of-Experts (MoE) architectures scale Large Language Models via expert specialization induced by conditional computation.<n>Some experts become functionally similar, while others functioning as de facto shared experts, limiting the effective capacity and model performance.<n>We propose Spectral-Decoupled MoE (SD-MoE), which decomposes both parameter and gradient in the spectral space.
  arXiv  Detail & Related papers  (2026-02-13T03:07:26Z)
• Expert Divergence Learning for MoE-based Language Models [21.857507958048696]
  We introduce Expert Divergence Learning, a novel pre-training strategy that explicitly encourages functional specialization among experts.<n>Our method incorporates a label-driven auxiliary loss that leverages domain labels inherent in pre-training corpora.<n>We validate our approach by pre-training MoE models of up to 15 billion parameters from scratch.
  arXiv  Detail & Related papers  (2026-02-10T06:58:38Z)
• MMedExpert-R1: Strengthening Multimodal Medical Reasoning via Domain-Specific Adaptation and Clinical Guideline Reinforcement [63.82954136824963]
  Medical Vision-Language Models excel at perception tasks with complex clinical reasoning required in real-world scenarios.<n>We propose a novel reasoning MedVLM that addresses these challenges through domain-specific adaptation and guideline reinforcement.
  arXiv  Detail & Related papers  (2026-01-16T02:32:07Z)
• Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations [48.890534958441016]
  This study investigates domain specialization and expert redundancy in large-scale MoE models.<n>We propose a simple yet effective pruning framework, EASY-EP, to identify and retain only the most relevant experts.<n>Experiments on DeepSeek-R1 and DeepSeek-V3-0324 show that our method can achieve comparable performances and $2.99times$ throughput under the same memory budget with full model with only half the experts.
  arXiv  Detail & Related papers  (2025-04-09T11:34:06Z)
• Convergence Rates for Softmax Gating Mixture of Experts [78.3687645289918]
  Mixture of experts (MoE) has emerged as an effective framework to advance the efficiency and scalability of machine learning models.<n>Central to the success of MoE is an adaptive softmax gating mechanism which takes responsibility for determining the relevance of each expert to a given input and then dynamically assigning experts their respective weights.<n>We perform a convergence analysis of parameter estimation and expert estimation under the MoE equipped with the standard softmax gating or its variants, including a dense-to-sparse gating and a hierarchical softmax gating.
  arXiv  Detail & Related papers  (2025-03-05T06:11:24Z)
• LoRKD: Low-Rank Knowledge Decomposition for Medical Foundation Models [59.961172635689664]
  "Knowledge Decomposition" aims to improve the performance on specific medical tasks. We propose a novel framework named Low-Rank Knowledge Decomposition (LoRKD) LoRKD explicitly separates gradients from different tasks by incorporating low-rank expert modules and efficient knowledge separation convolution.
  arXiv  Detail & Related papers  (2024-09-29T03:56:21Z)
• Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study [65.11303133775857]
  Mixture-of-Experts (MoE) computation amalgamates predictions from several specialized sub-models (referred to as experts) Sparse MoE selectively engages only a limited number, or even just one expert, significantly reducing overhead while empirically preserving, and sometimes even enhancing, performance.
  arXiv  Detail & Related papers  (2024-03-26T05:48:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.