Output Dilution: Redundant but Fragile Representations in MoE Models

• URL: http://arxiv.org/abs/2608.25231v1
• Date: Tue, 25 Aug 2026 23:41:07 GMT
• Title: Output Dilution: Redundant but Fragile Representations in MoE Models
• Abstract summary: Mixture-of-Experts (MoE) models appear to encode moral content as robustly as dense models, yet prove far more fragile in their encoding.<n>In OLMoE-1B-7B, linear probes recover moral valence from nearly every expert-layer combination, with mean peak-layer accuracy above 90%.<n>But these representations collapse under levels of activation noise that a dense model of matched size easily tolerates, with a 4.2-fold difference in robustness.<n>Because the MoE block averages across active experts before contributing to the residual stream, the feedforward signal reaching downstream layers is nearly two orders of magnitude smaller than in
• Score: 1.0829694003408499
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mixture-of-Experts (MoE) models appear to encode moral content as robustly as dense models, yet prove far more fragile in their encoding. In OLMoE-1B-7B, linear probes recover moral valence from nearly every expert-layer combination, with mean peak-layer accuracy above 90%. But these representations collapse under levels of activation noise that a dense model of matched size easily tolerates, with a 4.2-fold difference in robustness. We trace this to output dilution. Because the MoE block averages across active experts before contributing to the residual stream, the feedforward signal reaching downstream layers is nearly two orders of magnitude smaller than in a dense MLP. Moral information, our interest, survives aggregation intact but at a scale trivially overwhelmed by perturbation. Routing itself remains stable under noise while the vulnerability originates entirely in the diluted aggregate. Checkpoint trajectories confirm this is architectural, not learned. Experts never specialize and accuracy saturates within the first few thousand steps. In sparse architectures, redundant encoding does not imply robust encoding.

Related papers

• Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models [7.457366749109892]
  Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation.<n>DLMs offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refinement.<n>We systematically evaluate DLM robustness and calibration against AR baselines.
  arXiv  Detail & Related papers  (2026-07-29T18:46:43Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• MIRAGE: A Polarity-Flipping Encoding Subspace in LLM Agents [2.896184275229374]
  LLM agents are coerced into covertly encoding sensitive data.<n>The resulting outputs evade output-side detection but the underlying computation does not.<n>We build MIRAGE, a two-channel real-time monitor exploiting both signals.
  arXiv  Detail & Related papers  (2026-06-09T01:45:19Z)
• Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs [77.42033827176806]
  One-Shot Federated Learning, where a central server learns a global model in a single communication round, has emerged as a promising paradigm.<n>We propose a novel Federated Model Inversion and Token Relabel framework, which trains the global model by fully exploiting all patches of synthetic images.
  arXiv  Detail & Related papers  (2026-05-11T15:49:08Z)
• Layer Collapse in Diffusion Language Models [54.880703002010144]
  Diffusion language models (DLMs) have emerged as competitive alternatives to autoregressive (AR) language models.<n>We show that layer collapse in DLMs is not driven by undertraining but by overtraining.<n>Our findings have strong practical implications.
  arXiv  Detail & Related papers  (2026-05-07T14:39:40Z)
• From Early Encoding to Late Suppression: Interpreting LLMs on Character Counting Tasks [49.57538588967748]
  Large language models (LLMs) exhibit failures on elementary symbolic tasks such as character counting in a word, despite excelling on complex benchmarks.<n>We uncover a consistent phenomenon across modern architectures, including LLaMA, Qwen, and Gemma.<n>We show that symbolic reasoning failures in LLMs are not due to missing representations or insufficient scale, but arise from structured interference within the model's computation graph.
  arXiv  Detail & Related papers  (2026-04-01T11:40:12Z)
• Mixture-of-Depths Attention [65.80640499676542]
  Scaling depth is a key driver for large language models (LLMs)<n>We introduce mixture-of-depths attention (MoDA)<n>MoDA allows each attention head to attend to sequence KV pairs at the current layer and depth KV pairs from preceding layers.
  arXiv  Detail & Related papers  (2026-03-16T17:59:55Z)
• Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection [2.8547732086436306]
  A fundamental limitation of supervised deep learning is "Generalization Collapse"<n>We propose Latent Sculpting, a hierarchical two-stage representation learning framework.<n>We report an 88.89% detection rate on "Infiltration" scenarios.
  arXiv  Detail & Related papers  (2025-12-19T11:37:02Z)
• Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders [34.99839291352472]
  Multilayer perceptrons (MLPs) are integral part of large language models.<n>Recent methods learn interpretable approximations via neuron-level sparsity, yet fail to faithfully reconstruct the original mapping.<n>In this paper, we advocate for moving to layer-level sparsity to overcome the accuracy trade-off in sparse approximation.
  arXiv  Detail & Related papers  (2025-05-27T15:55:55Z)
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free [81.65559031466452]
  We conduct experiments to investigate gating-augmented softmax attention variants.<n>We find that a simple modification-applying a head-specific sigmoid gate after the Scaled Dot-Product Attention (SDPA)-consistently improves performance.
  arXiv  Detail & Related papers  (2025-05-10T17:15:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.