RouteScan: A Non-Intrusive Approach to Auditing MoE LLMs Safety via Expert Routing Telemetry

• URL: http://arxiv.org/abs/2605.24817v1
• Date: Sun, 24 May 2026 02:06:44 GMT
• Title: RouteScan: A Non-Intrusive Approach to Auditing MoE LLMs Safety via Expert Routing Telemetry
• Abstract summary: We propose RouteScan, a non-intrusive auditing framework for detecting harmful behaviors through expert routing telemetry.<n>We show that RouteScan achieves strong generalization, with an AUROC exceeding 0.93 on unseen harmful domains and 0.96 under novel jailbreak wrappers.
• Score: 24.677169755426906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mixture-of-Experts (MoE) architectures have become an increasingly important paradigm for scaling Large Language Models (LLMs). As MoE models are increasingly deployed in real-world services, safety auditing becomes necessary to verify whether these models produce or facilitate harmful behaviors during operation. However, existing content-based auditing methods typically require access to user prompts, model inputs, or generated outputs, potentially exposing sensitive user information and creating a fundamental tension between LLM safety and user privacy. On the other hand, we observe that, in MoE models, sparse expert routing maps different inputs to activate different expert-execution patterns, producing measurable footprints in low-level GPU execution telemetry. Inspired by this observation, we propose RouteScan, a non-intrusive auditing framework for detecting harmful behaviors through GPU-level expert routing telemetry. Specifically, RouteScan utilizes the number of active GPU threads allocated to expert modules during the prefilling phase as a discriminative micro-architectural fingerprint, and builds a lightweight detection pipeline that isolates cross-domain invariant risk indicators for the precise identification of malicious prompts. Comprehensive evaluations on open-source MoE LLMs with distinct routing designs demonstrate that RouteScan achieves strong generalization, with an AUROC exceeding 0.93 on unseen harmful domains and 0.96 under novel jailbreak wrappers. Moreover, empirical inversion tests show that the collected expert routing telemetry provides limited information for prompt reconstruction, suggesting a practical privacy advantage over content-based auditing methods.

Related papers

• Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signals [2.2302915692528367]
  We introduce InnerExpert, the first method to leverage MoE-specific signals for per-token hallucination detection.<n>Our results show that InnerExpert outperforms existing methods across five datasets and two MoE architectures.
  arXiv  Detail & Related papers  (2026-08-18T12:00:20Z)
• Understanding Safety-Sensitive Expert Behavior in Mixture-of-Experts LLMs [21.519746366998646]
  A common intuition is that safety behavior may be controlled by routing harmful requests to distinct refusal-oriented experts.<n>We present **RASET**, a framework that probes safety enforcement that is localized in a small subset of experts.<n>Results reveal a distinct MoE safety risk, highlighting the need for expert-aware alignment mechanisms.
  arXiv  Detail & Related papers  (2026-05-28T10:09:51Z)
• $D^2$-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation-Aware Routing [63.49501120848927]
  Safety monitoring for large language models (D-LLMs) remains largely unexplored.<n>We propose $D2$-Monitor, a bi-level safety monitor for D-LLMs.<n>$D2$-Monitor adopts a lightweight probe as an always-on monitor to jointly estimate hesitation and perform base classification.
  arXiv  Detail & Related papers  (2026-05-25T14:22:21Z)
• Rethinking Side-Channel Analysis: Automated Discovery and Analysis of Side-Channel Leakage with LLM-Assisted Agents [25.6110029853065]
  We present SCAgent, an automated framework for side-channel risk analysis.<n>SCAgent identifies sensitive targets beyond manually specified events.<n>We instantiate SCAgent on iOS as a first step, focusing on OS-level side channels observable by unprivileged applications.
  arXiv  Detail & Related papers  (2026-05-17T12:03:45Z)
• TraceScope: Interactive URL Triage via Decoupled Checklist Adjudication [14.375064108289115]
  We present TraceScope, a decoupled triage pipeline that operationalizes this workflow at scale.<n>To prevent the observer effect and ensure safety, a sandboxed operator agent drives a real GUI browser guided by visual motivation to elicit page behavior.<n>An adjudicator agent circumvents LLM context limitations by querying evidence on demand to verify a MITRE ATT&CK checklist, and generates an audit-ready report with extracted indicators of compromise (IOCs)<n>Our evaluation reveals that TraceScope demonstrates superior performance in a real-world scenario as well, successfully detecting sophisticated phishing attempts that current state-of-the-art defenses fail to identify
  arXiv  Detail & Related papers  (2026-04-23T16:31:42Z)
• The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search [58.8834056209347]
  Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs.<n>We introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model's knowledge base.
  arXiv  Detail & Related papers  (2025-12-01T07:05:23Z)
• RouteMark: A Fingerprint for Intellectual Property Attribution in Routing-based Model Merging [69.2230254959204]
  We propose RouteMark, a framework for IP protection in merged MoE models.<n>Our key insight is that task-specific experts exhibit stable and distinctive routing behaviors under probing inputs.<n>For attribution and tampering detection, we introduce a similarity-based matching algorithm.
  arXiv  Detail & Related papers  (2025-08-03T14:51:58Z)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents [52.92354372596197]
  Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities.<n>This interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior.<n>We propose a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control and data-level constraints.
  arXiv  Detail & Related papers  (2025-06-13T05:01:09Z)
• Defending against Indirect Prompt Injection by Instruction Detection [109.30156975159561]
  InstructDetector is a novel detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks.<n>InstructDetector achieves a detection accuracy of 99.60% in the in-domain setting and 96.90% in the out-of-domain setting, and reduces the attack success rate to just 0.03% on the BIPIA benchmark.
  arXiv  Detail & Related papers  (2025-05-08T13:04:45Z)
• How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities [62.474732677086855]
  Large language model (LLM) routing has emerged as a crucial strategy for balancing computational costs with performance.<n>We propose the DSC benchmark: Diverse, Simple, and Categorized, an evaluation framework that categorizes router performance across a broad spectrum of query types.
  arXiv  Detail & Related papers  (2025-03-20T19:52:30Z)
• Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences [18.36319991890607]
  We present Wildflare GuardRail, a guardrail pipeline designed to enhance the safety and reliability of Large Language Model (LLM) inferences.<n>Wildflare GuardRail integrates several core functional modules, including Safety Detector that identifies unsafe inputs and detects hallucinations in model outputs.<n>The lightweight wrappers can address malicious URLs in model outputs in 1.06s per query with 100% accuracy without costly model calls.
  arXiv  Detail & Related papers  (2025-02-12T05:48:57Z)
• Soft Merging of Experts with Adaptive Routing [38.962451264172856]
  We introduce Soft Merging of Experts with Adaptive Routing (SMEAR) SMEAR avoids discrete routing by using a single "merged" expert constructed via a weighted average of all of the experts' parameters. We empirically validate that models using SMEAR outperform models that route based on metadata or learn sparse routing through gradient estimation.
  arXiv  Detail & Related papers  (2023-06-06T15:04:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.