Understanding Safety-Sensitive Expert Behavior in Mixture-of-Experts LLMs

• URL: http://arxiv.org/abs/2605.29708v1
• Date: Thu, 28 May 2026 10:09:51 GMT
• Title: Understanding Safety-Sensitive Expert Behavior in Mixture-of-Experts LLMs
• Abstract summary: A common intuition is that safety behavior may be controlled by routing harmful requests to distinct refusal-oriented experts.<n>We present **RASET**, a framework that probes safety enforcement that is localized in a small subset of experts.<n>Results reveal a distinct MoE safety risk, highlighting the need for expert-aware alignment mechanisms.
• Score: 21.519746366998646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mixture-of-Experts (MoE) LLMs rely on sparse, router-driven expert activation, yet how safety alignment interacts with routed expert specialization remains underexplored. A common intuition is that safety behavior may be controlled by routing harmful requests to distinct refusal-oriented experts. In this work, we provide empirical evidence for a different picture: routing patterns in aligned MoE LLMs are largely topic-driven, while safety behavior can be altered with little change to the model's intrinsic routing path. Motivated by this observation, we present **RASET** (**R**outer-**A**gnostic **S**afety-critical **E**xpert **T**uning), a red-teaming framework that probes safety enforcement that is localized in a small subset of experts while preserving the model's intrinsic routing behavior. **RASET** identifies safety-critical experts via a contrastive routing-sensitivity criterion and applies parameter-efficient tuning only to the selected experts, minimizing semantic disruption relative to router-steering interventions. These results reveal a distinct MoE safety risk, highlighting the need for expert-aware alignment mechanisms.

Related papers

• AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectories [12.694014584212695]
  We introduce AURA-Eval, a framework combining controlled augmentation with granular diagnosis of behavior in tool-use trajectories.<n>Using 157 sourced trajectories, we generate 1,249 evaluation items and evaluate 20 frontier and open-weight models.<n>Our results show that LLM agents engage in unsafe behavior more often when no safe path exists.
  arXiv  Detail & Related papers  (2026-09-06T18:58:56Z)
• HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models [28.315229912829253]
  HiRoute is an input-adaptive hierarchical prompt-tuning framework for large language models.<n>It separates category-agnostic safety control from category-specific response guidance.<n>Experiments show that HiRoute achieves high safety rates across multiple safety benchmarks.
  arXiv  Detail & Related papers  (2026-08-13T04:49:51Z)
• AgentLens: Interpretable Safety Steering via Mechanistic Subspaces for Multi-Turn Coding Agent [53.82005364479556]
  Coding agents based on large language models (LLMs) demonstrate remarkable autonomous capabilities.<n>Existing safety mechanisms mainly rely on external guardrails.<n>We propose AgentLens, a white-box defense framework that performs runtime safety detection and representation-level mitigation.
  arXiv  Detail & Related papers  (2026-06-21T21:13:55Z)
• RouteScan: A Non-Intrusive Approach to Auditing MoE LLMs Safety via Expert Routing Telemetry [24.677169755426906]
  We propose RouteScan, a non-intrusive auditing framework for detecting harmful behaviors through expert routing telemetry.<n>We show that RouteScan achieves strong generalization, with an AUROC exceeding 0.93 on unseen harmful domains and 0.96 under novel jailbreak wrappers.
  arXiv  Detail & Related papers  (2026-05-24T02:06:44Z)
• RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Models [10.741523413040559]
  We propose RASA, a routing-aware expert-level alignment framework.<n> RASA identifies experts disproportionately activated by successful jailbreaks, selectively fine-tunes only these experts under fixed routing.<n>Our results suggest that robust MoE safety alignment benefits from targeted expert repair rather than global parameter updates.
  arXiv  Detail & Related papers  (2026-02-04T11:19:15Z)
• SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning [83.66308307152808]
  We propose StAbilized Mixture-of-Experts (SAME) for Multimodal Continual Instruction Tuning (MCIT)<n>SAME stabilizes expert selection by decomposing routing dynamics into subspaces and updating only task-relevant directions.<n>It also introduces adaptive expert activation to freeze selected experts during training, reducing redundant and cross-task interference.
  arXiv  Detail & Related papers  (2026-02-02T11:47:06Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process [66.38541693477181]
  We propose an unsupervised framework for discovering reasoning vectors, which we define as directions in the activation space that encode distinct reasoning behaviors.<n>By segmenting chain-of-thought traces into sentence-level'steps', we uncover disentangled features corresponding to interpretable behaviors such as reflection and backtracking.<n>We demonstrate the ability to control response confidence by identifying confidence-related vectors in the SAE decoder space.
  arXiv  Detail & Related papers  (2025-12-30T05:09:11Z)
• D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models [62.83226685925107]
  Deceptive Reasoning Exposure Suite (D-REX) is a novel dataset designed to evaluate the discrepancy between a model's internal reasoning process and its final output.<n>Each sample in D-REX contains the adversarial system prompt, an end-user's test query, the model's seemingly innocuous response, and, crucially, the model's internal chain-of-thought.<n>We demonstrate that D-REX presents a significant challenge for existing models and safety mechanisms.
  arXiv  Detail & Related papers  (2025-09-22T15:59:40Z)
• Steering MoE LLMs via Expert (De)Activation [118.23403783503026]
  Mixture-of-Experts (MoE) in Large Language Models (LLMs) routes each token through a subset of specialized Feed-Forward Networks (FFN)<n>We present SteerMoE, a framework for steering MoE models by detecting and controlling behavior-linked experts.
  arXiv  Detail & Related papers  (2025-09-11T17:55:09Z)
• RouteMark: A Fingerprint for Intellectual Property Attribution in Routing-based Model Merging [69.2230254959204]
  We propose RouteMark, a framework for IP protection in merged MoE models.<n>Our key insight is that task-specific experts exhibit stable and distinctive routing behaviors under probing inputs.<n>For attribution and tampering detection, we introduce a similarity-based matching algorithm.
  arXiv  Detail & Related papers  (2025-08-03T14:51:58Z)
• SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification [27.13765502389987]
  MoE-specific safety risk of positional vulnerability is formalized and systematically analyzed.<n>An analytical framework, SAFEx, is presented to robustly identify, characterize, and validate safety-critical experts.<n>Expert-level interventions are conducted to probe causality and to test mitigation.
  arXiv  Detail & Related papers  (2025-06-20T15:09:10Z)
• How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities [62.474732677086855]
  Large language model (LLM) routing has emerged as a crucial strategy for balancing computational costs with performance.<n>We propose the DSC benchmark: Diverse, Simple, and Categorized, an evaluation framework that categorizes router performance across a broad spectrum of query types.
  arXiv  Detail & Related papers  (2025-03-20T19:52:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.