Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE

• URL: http://arxiv.org/abs/2606.26938v1
• Date: Thu, 25 Jun 2026 12:13:40 GMT
• Title: Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE
• Abstract summary: Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation.<n>We propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism.<n>By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance.
• Score: 49.45064510462232
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve efficiency and performance. However, we identify a routing assignment problem in existing diffusion MoE frameworks: the router fails to accurately allocate more computational resources to salient tokens. Our analysis attributes this failure to the router's reliance on noise-corrupted latent features throughout the denoising process. Such stochastic noise obscures the critical structural and textural information, thereby preventing the router from effectively distinguishing salient tokens. To address this, we propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism, which utilizes clean latent features as a noise-free guidance signal for routing. By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance, enabling the identification of salient tokens even in high-noise stages. Furthermore, we introduce a trajectory routing loss to constrain the compute allocation throughout the multi-step denoising trajectory, ensuring precise resource allocation along the generation rollout. Extensive experiments demonstrate that SharpMoE serves as a versatile, plug-and-play solution that further enhances the pretrained, converged MoE models, achieving state-of-the-art performance in visual generation.

Related papers

• CRISP: Cliff-awaRe Input-adaptive Sparse Prefilling with Structural-Mass-Motivated Routing [90.63350360476294]
  We present CRISP (Cliff-awaRe Input-adaptive Sparse Prefilling), which identifies and addresses two structural challenges in this dynamic routing paradigm.<n>We show that the routing decision can be read directly off the structure of the proxy attention map.<n>We formalize the post-softmax mass cliff and demonstrate theoretically that strictly cumulative coverage thresholds accumulate O(n) background noise at long contexts.
  arXiv  Detail & Related papers  (2026-09-01T22:49:46Z)
• TRaM-VSR: Importance-Aware Token Routing and Merging for One-Step Diffusion Video Super-Resolution [52.96050253456634]
  TRaM-VSR is a framework for adaptive token allocation using context-aware video priors and network-levels.<n>It is shown that TRaM-VSR accelerates inference significantly while preserving state-of-the-art reconstruction quality and robust temporal consistency.
  arXiv  Detail & Related papers  (2026-07-24T11:57:26Z)
• When Model Merging Breaks Routing: Training-Free Calibration for MoE [67.27839627141972]
  Hessian-Aware Router (HARC) is a training-free framework that leverages second-order curvature information to realign the merged router.<n> Experiments on mathematical reasoning and code generation tasks show that HARC effectively mitigates routing breakdown across diverse MoE merging baselines.
  arXiv  Detail & Related papers  (2026-06-02T09:33:33Z)
• Learned Relay Representations for Forward-Thinking Discrete Diffusion Models [34.17541648016911]
  We propose Learned Relay Representations (Relay) to avoid a hard reset between denoising rounds.<n>Relay introduces a differentiable per-token channel that passes information between forward passes and is trained via truncated backpropagation through time.<n>We show that Relay can be scaled to state-of-the-art Diffusion Language Models (DLMs) and is seamlessly compatible with techniques like block diffusion and KV caching.
  arXiv  Detail & Related papers  (2026-05-21T18:53:22Z)
• TrafficMoE: Heterogeneity-aware Mixture of Experts for Encrypted Traffic Classification [12.748275751587363]
  Encrypted traffic classification is a critical task for network security.<n>The one-size-fits-all static design is inherently flawed.<n>We propose TrafficMoE, a framework that breaks through the bottleneck of static modeling.
  arXiv  Detail & Related papers  (2026-03-31T10:05:54Z)
• TraceRouter: Robust Safety for Large Foundation Models via Path-Level Intervention [44.64827167753535]
  harmful semantics act as distributed, cross-layer circuits, rendering localized interventions brittle and detrimental to utility.<n>We propose textbfTrace, a path-level framework that traces and disconnects the causal propagation circuits of illicit semantics.<n>Trace significantly outperforms state-of-the-art baselines, achieving a superior trade-off between adversarial robustness and general utility.
  arXiv  Detail & Related papers  (2026-01-29T15:58:12Z)
• Lost in the Noise: How Reasoning Models Fail with Contextual Distractors [57.31788955167306]
  Recent advances in reasoning models and agentic AI systems have led to an increased reliance on diverse external information.<n>We introduce NoisyBench, a comprehensive benchmark that systematically evaluates model robustness across 11 datasets in RAG, reasoning, alignment, and tool-use tasks.<n>Our evaluation reveals a catastrophic performance drop of up to 80% in state-of-the-art models when faced with contextual distractors.
  arXiv  Detail & Related papers  (2026-01-12T05:43:51Z)
• Routing Matters in MoE: Scaling Diffusion Transformers with Explicit Routing Guidance [79.21541758879012]
  Mixture-of-Experts (MoE) has emerged as a powerful paradigm for scaling model capacity while preserving computational efficiency.<n>We present ProMoE, an MoE framework featuring a two-step router with explicit routing guidance that promotes expert specialization.
  arXiv  Detail & Related papers  (2025-10-28T17:59:02Z)
• DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers [86.5541501589166]
  DiffMoE is a batch-level global token pool that enables experts to access global token distributions during training.<n>It achieves state-of-the-art performance among diffusion models on ImageNet benchmark.<n>The effectiveness of our approach extends beyond class-conditional generation to more challenging tasks such as text-to-image generation.
  arXiv  Detail & Related papers  (2025-03-18T17:57:07Z)
• ADRN: Attention-based Deep Residual Network for Hyperspectral Image Denoising [52.01041506447195]
  We propose an attention-based deep residual network to learn a mapping from noisy HSI to the clean one. Experimental results demonstrate that our proposed ADRN scheme outperforms the state-of-the-art methods both in quantitative and visual evaluations.
  arXiv  Detail & Related papers  (2020-03-04T08:36:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.