Divide, Weight, and Route: Difficulty-Aware Optimization with Dynamic Expert Fusion for Long-tailed Recognition

• URL: http://arxiv.org/abs/2508.19630v1
• Date: Wed, 27 Aug 2025 07:09:00 GMT
• Title: Divide, Weight, and Route: Difficulty-Aware Optimization with Dynamic Expert Fusion for Long-tailed Recognition
• Abstract summary: We propose textbfDQRoute, a modular framework that combines difficulty-aware optimization with dynamic expert collaboration.<n>DQRoute first estimates class-wise difficulty based on prediction uncertainty and historical performance.<n>At inference time, expert predictions are weighted by confidence scores derived from expert-specific OOD detectors, enabling input-adaptive routing.
• Score: 3.616706632062255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-tailed visual recognition is challenging not only due to class imbalance but also because of varying classification difficulty across categories. Simply reweighting classes by frequency often overlooks those that are intrinsically hard to learn. To address this, we propose \textbf{DQRoute}, a modular framework that combines difficulty-aware optimization with dynamic expert collaboration. DQRoute first estimates class-wise difficulty based on prediction uncertainty and historical performance, and uses this signal to guide training with adaptive loss weighting. On the architectural side, DQRoute employs a mixture-of-experts design, where each expert specializes in a different region of the class distribution. At inference time, expert predictions are weighted by confidence scores derived from expert-specific OOD detectors, enabling input-adaptive routing without the need for a centralized router. All components are trained jointly in an end-to-end manner. Experiments on standard long-tailed benchmarks demonstrate that DQRoute significantly improves performance, particularly on rare and difficult classes, highlighting the benefit of integrating difficulty modeling with decentralized expert routing.

Related papers

• Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation [49.8459545291965]
  Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts.<n>We study this dependency by decomposing sparsely upcycled feed-forward experts into key-value channels.<n>We instantiate it in UniF-MoE, a unified framework for token-adaptive MoE computation.
  arXiv  Detail & Related papers  (2026-08-11T02:40:58Z)
• Elbow-Based MoE Routing: A Training-Free Inference Time Plugin for Expert Selection [0.0]
  Mixture-of-Experts (MoE) models enable model scaling while maintaining low inference-time compute.<n>We introduce elbow-based routing, a training-free inference-time modification that dynamically adjusts the number of experts on a per-token basis.<n> Experiments on a state-of-the-art MoE model demonstrate an average latency reduction of 5.3% while maintaining accuracy across six benchmarks.
  arXiv  Detail & Related papers  (2026-08-05T03:10:17Z)
• Teacher-Guided Routing for Sparse Vision Mixture-of-Experts [25.423413775524704]
  We propose TGR-MoE: Teacher-Guided Routing for Sparse Vision Mixture-of-Experts.<n>It stabilizes router learning using supervision derived from a pretrained dense teacher model.<n>It consistently improves both accuracy and routing consistency, while maintaining stable training even under highly sparse configurations.
  arXiv  Detail & Related papers  (2026-04-23T06:34:10Z)
• Multi-Domain Learning with Global Expert Mapping [102.62297074508147]
  Mixture-of-Experts (MoE) models provide a scalable solution by routing inputs to specializedworks (experts)<n>We propose GEM (Global Expert Mapping), a planner-compiler framework that replaces the learned router with a global scheduler.<n>Our planner, based on linear programming relaxation, computes a fractional assignment of datasets to experts, while the compiler applies hierarchical rounding to convert this soft plan into a deterministic, capacity-aware mapping.<n>Experiments show that GEM-DINO achieves state-of-the-art performance on the UODB benchmark, with notable gains on underrepresented datasets and solves task interference in few
  arXiv  Detail & Related papers  (2026-04-20T21:09:34Z)
• Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts [56.02203242609604]
  Large language models (LLMs) fine-tuned with lightweight adapters achieve strong performance across diverse tasks.<n>Fusing independently trained models with different strengths has shown promise for multi-task learning through three main strategies.<n>We empirically evaluate their trade-offs, addressing two key questions: What are the advantages of going beyond uniform ensembling or merging? and does the flexibility of routing justify its complexity?
  arXiv  Detail & Related papers  (2026-03-03T21:44:11Z)
• SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning [83.66308307152808]
  We propose StAbilized Mixture-of-Experts (SAME) for Multimodal Continual Instruction Tuning (MCIT)<n>SAME stabilizes expert selection by decomposing routing dynamics into subspaces and updating only task-relevant directions.<n>It also introduces adaptive expert activation to freeze selected experts during training, reducing redundant and cross-task interference.
  arXiv  Detail & Related papers  (2026-02-02T11:47:06Z)
• ERMoE: Eigen-Reparameterized Mixture-of-Experts for Stable Routing and Interpretable Specialization [13.182475975397251]
  ERMoE is a sparse MoE transformer that replaces learned gating logits with an "Eigenbasis Score"<n>We show that ERMoE achieves state-of-the-art accuracy on ImageNet classification and cross-modal image-text retrieval benchmarks.<n>A 3D MRI variant (ERMoE-ba) improves brain age prediction accuracy by more than 7% and yields interpretable expert specializations.
  arXiv  Detail & Related papers  (2025-11-14T05:31:37Z)
• Staying in the Sweet Spot: Responsive Reasoning Evolution via Capability-Adaptive Hint Scaffolding [59.60915947702282]
  Reinforcement learning with verifiable rewards (RLVR) has achieved remarkable success in enhancing the reasoning capabilities of large language models (LLMs)<n>Existing RLVR methods often suffer from exploration inefficiency due to mismatches between the training data's difficulty and the model's capability.<n>We propose SEELE, a novel supervision-aided RLVR framework that dynamically adjusts problem difficulty to stay within the high-efficiency region.
  arXiv  Detail & Related papers  (2025-09-08T17:36:21Z)
• Divide, Specialize, and Route: A New Approach to Efficient Ensemble Learning [0.0]
  We propose Hellsemble, a novel ensemble framework for binary classification.<n>Hellsemble incrementally partitions the dataset into circles of difficulty.<n>It achieves strong classification accuracy while maintaining computational efficiency and interpretability.
  arXiv  Detail & Related papers  (2025-06-25T20:26:04Z)
• Advancing Expert Specialization for Better MoE [22.570561334474252]
  Mixture-of-Experts (MoE) models enable efficient scaling of large language models (LLMs) by activating only a subset of experts per input.<n>We observe that the commonly used auxiliary load balancing loss often leads to expert overlap and overly uniform routing.<n>We propose a simple yet effective solution that introduces two complementary objectives.
  arXiv  Detail & Related papers  (2025-05-28T13:09:47Z)
• On the Role of Discrete Representation in Sparse Mixture of Experts [33.809432499123275]
  We propose a new architecture dubbed Vector-Quantized Mixture of Experts (VQMoE)<n>VQMoE is an effective solution for scaling up model capacity without increasing the computational costs.<n>We show that VQMoE achieves a 28% improvement in routers compared to other SMoE routing methods.
  arXiv  Detail & Related papers  (2024-11-28T22:32:01Z)
• Complexity Experts are Task-Discriminative Learners for Any Image Restoration [80.46313715427928]
  We introduce complexity experts" -- flexible expert blocks with varying computational complexity and receptive fields.<n>This preference effectively drives task-specific allocation, assigning tasks to experts with the appropriate complexity.<n>The proposed MoCE-IR model outperforms state-of-the-art methods, affirming its efficiency and practical applicability.
  arXiv  Detail & Related papers  (2024-11-27T15:58:07Z)
• Harder Tasks Need More Experts: Dynamic Routing in MoE Models [58.18526590138739]
  We introduce a novel dynamic expert selection framework for Mixture of Experts (MoE) models. Our method dynamically selects experts based on the confidence level in expert selection for each input.
  arXiv  Detail & Related papers  (2024-03-12T13:41:15Z)
• Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization [51.98792406392873]
  Mixture of Experts (MoE) provides a powerful way to decompose dense layers into smaller, modular computations. A major challenge lies in the computational cost of scaling the number of experts high enough to achieve fine-grained specialization. We propose the Multilinear Mixture of Experts ($mu$MoE) layer to address this, focusing on vision models.
  arXiv  Detail & Related papers  (2024-02-19T21:20:22Z)
• Soft Merging of Experts with Adaptive Routing [38.962451264172856]
  We introduce Soft Merging of Experts with Adaptive Routing (SMEAR) SMEAR avoids discrete routing by using a single "merged" expert constructed via a weighted average of all of the experts' parameters. We empirically validate that models using SMEAR outperform models that route based on metadata or learn sparse routing through gradient estimation.
  arXiv  Detail & Related papers  (2023-06-06T15:04:31Z)
• On the Representation Collapse of Sparse Mixture of Experts [102.83396489230375]
  Sparse mixture of experts provides larger model capacity while requiring a constant computational overhead. It employs the routing mechanism to distribute input tokens to the best-matched experts according to their hidden representations. However, learning such a routing mechanism encourages token clustering around expert centroids, implying a trend toward representation collapse.
  arXiv  Detail & Related papers  (2022-04-20T01:40:19Z)
• Long-tailed Recognition by Routing Diverse Distribution-Aware Experts [64.71102030006422]
  We propose a new long-tailed classifier called RoutIng Diverse Experts (RIDE) It reduces the model variance with multiple experts, reduces the model bias with a distribution-aware diversity loss, reduces the computational cost with a dynamic expert routing module. RIDE outperforms the state-of-the-art by 5% to 7% on CIFAR100-LT, ImageNet-LT and iNaturalist 2018 benchmarks.
  arXiv  Detail & Related papers  (2020-10-05T06:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.