MASCing: Configurable Mixture-of-Experts Behavior via Activation Steering Masks

• URL: http://arxiv.org/abs/2604.27818v1
• Date: Thu, 30 Apr 2026 12:58:57 GMT
• Title: MASCing: Configurable Mixture-of-Experts Behavior via Activation Steering Masks
• Abstract summary: We present MASCing, a framework that enables flexible reconfiguration of MoE behavior across diverse safety scenarios without retraining.<n> MASCing uses an LSTM-based surrogate model to capture cross-layer routing dependencies and map routing logits to downstream behaviors.<n>It then optimize a steering matrix to identify behavior-relevant expert circuits and apply steering masks to the routing gates to override expert selection.
• Score: 12.70638634576904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mixture-of-Experts (MoE) architectures in Large Language Models (LLMs) have significantly reduced inference costs through sparse activation. However, this sparse activation paradigm also introduces new safety challenges. Since only a subset of experts is engaged for each input, model behavior becomes coupled to routing decisions, yielding a difficult-to-control mechanism that can vary across safety-relevant scenarios. At the same time, adapting model behavior through full fine-tuning or retraining is costly, especially when developers need to rapidly configure the same model for different safety objectives. We present MASCing (MoE Activation Steering Configuration), the first framework that enables flexible reconfiguration of MoE behavior across diverse safety scenarios without retraining. MASCing uses an LSTM-based surrogate model to capture cross-layer routing dependencies and map routing logits to downstream behaviors. It then optimizes a steering matrix to identify behavior-relevant expert circuits and, at inference time, applies steering masks to the routing gates to override expert selection. This enables targeted enhancement or suppression of specific behaviors while preserving general language utility. To demonstrate its reconfigurability, we apply MASCing to two different safety-related objectives and observe consistent gains with negligible overhead across seven open-source MoE models. For multi-turn jailbreak defense, it improves the average defense success rate from 52.5% to 83.9%, with gains of up to 89.2%. For adult-content generation, MASCing enables models to comply with such requests that would otherwise be refused, increasing the average generation success rate from 52.6% to 82.0%, with gains of up to 93.0%. These results establish MASCing as a practical, lightweight, and flexible framework for scenario-specific safety reconfiguration in MoE models.

Related papers

• RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models [13.298401993417611]
  We introduce a router-agnostic representation engineering framework for MoE language models.<n>RARE projects arbitrary behavioral perturbations onto the null space of the router matrix, thereby removing router-visible components.<n>On harmfulness steering, RARE reaches an average attack success rate of 53.3% while retaining 67.8% MMLU accuracy.
  arXiv  Detail & Related papers  (2026-08-21T15:45:36Z)
• MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models [53.05463623673949]
  MaskForge is a fully black-box adaptive attack that casts dLLM red-teaming as optimized search over a growing library of structural patterns.<n>It achieves an average attack success rate of 79.3%, a 17.6% improvement over the strongest competing dLLM baseline.
  arXiv  Detail & Related papers  (2026-06-01T18:10:21Z)
• MESA: Improving MoE Safety Alignment via Decentralized Expertise [55.297863000560596]
  MESA (MoE Safety Alignment) is a targeted alignment framework for MoE-based Large Language Models.<n>It strategically decentralizes safety responsibility to maximize coverage while minimizing interference with utility.<n>MESA achieves robust defensive performance against varied harmful benchmarks while preserving helpfulness.
  arXiv  Detail & Related papers  (2026-05-30T09:54:38Z)
• RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs [9.24144060095413]
  We present RouteHijack, a routing-aware jailbreak for large language models (LLMs)<n> RouteHijack performs response-driven expert localization to identify safety-critical and harmful experts.<n>It then constructs adversarial suffixes with a routing-aware objective that suppresses safety experts, promotes harmful experts, and prevents early-stage refusal.
  arXiv  Detail & Related papers  (2026-05-01T11:54:55Z)
• Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use [6.622648583261088]
  Agentic language models must plan, call tools, and execute long-horizon actions where a single misstep can cause irreversible harm.<n>We introduce MOSAIC, a framework that aligns agents for safe multi-step tool use by making safety decisions explicit and learnable.<n>We show that MOSAIC reduces harmful behavior by up to 50%, increases harmful-task refusal by over 20% on injection attacks, cuts privacy leakage, and preserves or improves benign task performance.
  arXiv  Detail & Related papers  (2026-03-03T17:59:35Z)
• Large Language Lobotomy: Jailbreaking Mixture-of-Experts via Expert Silencing [14.891975420982504]
  We propose Large Language Lobotomy (L$3$), a training-free, architecture-agnostic attack that compromises safety alignment by exploiting expert routing dynamics.<n>L$3$ learns routing patterns that correlate with refusal, attributes safety behavior to specific experts, and adaptively silences the most safety-relevant experts until harmful outputs are produced.<n>We evaluate L$3$ on eight state-of-the-art open-source MoE LLMs and show that our adaptive expert silencing increases average attack success from 7.3% to 70.4%, reaching up to 86.3%, outperforming prior training-free
  arXiv  Detail & Related papers  (2026-02-09T14:42:11Z)
• Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models [62.16655896700062]
  Activation steering is a technique to enhance the utility of Large Language Models (LLMs)<n>We show that it unintentionally introduces critical and under-explored safety risks.<n>Experiments reveal that these interventions act as a force multiplier, creating new vulnerabilities to jailbreaks and increasing attack success rates to over 80% on standard benchmarks.
  arXiv  Detail & Related papers  (2026-02-03T12:32:35Z)
• GSAE: Graph-Regularized Sparse Autoencoders for Robust LLM Safety Steering [5.124731939041066]
  Large language models (LLMs) face critical safety challenges as they can be manipulated to generate harmful content through adversarial prompts and jailbreak attacks.<n>We introduce Graph-Regularized Sparse Autoencoders (GSAEs), which extends SAEs with a Laplacian smoothness penalty on the neuron co-activation graph.<n>We demonstrate that GSAE enables effective safety steering, assembling features into a weighted set of safety-relevant directions and controlling them with a two-stage gating mechanism.
  arXiv  Detail & Related papers  (2025-12-07T04:46:30Z)
• Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers [12.47462301643593]
  Large language models (LLMs) with Mixture-of-Experts (MoE) architectures achieve impressive performance and efficiency by dynamically routing inputs to specializedworks, known as experts.<n>We propose BadSwitch, a novel backdoor framework that integrates task-coupled dynamic trigger optimization with a sensitivity-guided Top-S expert tracing mechanism.
  arXiv  Detail & Related papers  (2025-10-15T12:11:02Z)
• Steering MoE LLMs via Expert (De)Activation [118.23403783503026]
  Mixture-of-Experts (MoE) in Large Language Models (LLMs) routes each token through a subset of specialized Feed-Forward Networks (FFN)<n>We present SteerMoE, a framework for steering MoE models by detecting and controlling behavior-linked experts.
  arXiv  Detail & Related papers  (2025-09-11T17:55:09Z)
• Moshi Moshi? A Model Selection Hijacking Adversarial Attack [16.020656810797256]
  We present MOSHI, the first adversarial attack specifically targeting model selection.<n>Our attack causes an average 88.30% decrease in generalization capabilities, an 83.33% increase in latency, and an increase of up to 105.85% in energy consumption.<n>These results highlight the significant vulnerabilities in model selection processes and their potential impact on real-world applications.
  arXiv  Detail & Related papers  (2025-02-20T14:16:15Z)
• Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast [58.98411447739218]
  Mixture-of-Experts (MoE) has emerged as a prominent architecture for scaling model size while maintaining computational efficiency. We propose Self-Contrast Mixture-of-Experts (SCMoE), a training-free strategy that utilizes unchosen experts in a self-contrast manner during inference. Our method is conceptually simple and computationally lightweight, as it incurs minimal latency compared to greedy decoding.
  arXiv  Detail & Related papers  (2024-05-23T12:45:29Z)
• Masked Autoencoding for Scalable and Generalizable Decision Making [93.84855114717062]
  MaskDP is a simple and scalable self-supervised pretraining method for reinforcement learning and behavioral cloning. We find that a MaskDP model gains the capability of zero-shot transfer to new BC tasks, such as single and multiple goal reaching.
  arXiv  Detail & Related papers  (2022-11-23T07:04:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.