WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

• URL: http://arxiv.org/abs/2607.13099v1
• Date: Tue, 14 Jul 2026 06:18:05 GMT
• Title: WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency
• Abstract summary: Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse.<n>WaterMoE embeds watermarking signals through controlled perturbation into the expert selection at each router.<n>WaterMoE consistently outperforms state-of-the-art watermarking methods on a benchmark.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse, motivating the need for reliable watermarking techniques. However, these techniques have rarely been adopted in practice mainly for two reasons: i) severely degraded model performance, and ii) additional inference overhead. To confirm the problem, we construct a comprehensive benchmark spanning different generation tasks to systematically evaluate 9 representative watermarking methods. We found almost all existing methods are designed for text fluency, but not for restricted and complicated tasks, and their overhead prevents them from deployment in latency-critical systems. To address i) and ii), we propose an LLM watermarking scheme \textit{WaterMoE} for the growingly popular Mixture-of-Experts (MoE) LLMs. WaterMoE embeds watermarking signals through controlled perturbation into the expert selection at each router, which accumulates to token selection shift at the final output. In contrast to watermarking as a post-processing token-sampling approach, WaterMoE embeds watermark within the inference loop incurring negligible quality degradation and computational overhead. Extensive experiments demonstrate that our method achieves a fidelity performance close to the unwatermarked and consistently outperforms state-of-the-art watermarking methods on the benchmark, with up to $4\times$ speedup, incurring merely 1\% additional inference latency compared to native generation. The results demonstrate the capability of WaterMoE to be deployed in real-world tasks.

Related papers

• WaterSearch: A Quality-Aware Search-based Watermarking Framework for Large Language Models [17.137667672391725]
  WaterSearch is a sentence-level, search-based watermarking framework.<n>WaterSearch enhances text quality by jointly optimizing two key aspects: 1) distribution fidelity and 2) watermark signal characteristics.<n>Our method achieves an average performance improvement of 51.01% over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-11-30T11:11:21Z)
• Character-Level Perturbations Disrupt LLM Watermarks [64.60090923837701]
  We formalize the system model for Large Language Model (LLM) watermarking.<n>We characterize two realistic threat models constrained on limited access to the watermark detector.<n>We demonstrate character-level perturbations are significantly more effective for watermark removal under the most restrictive threat model.<n> Experiments confirm the superiority of character-level perturbations and the effectiveness of the Genetic Algorithm (GA) in removing watermarks under realistic constraints.
  arXiv  Detail & Related papers  (2025-09-11T02:50:07Z)
• Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models [63.450843788680196]
  We show that it is impossible to simultaneously maintain the highest watermark strength and the highest sampling efficiency. We propose two methods that maintain either the sampling efficiency or the watermark strength, but not both. Our work provides a rigorous theoretical foundation for understanding the inherent trade-off between watermark strength and sampling efficiency.
  arXiv  Detail & Related papers  (2024-10-27T12:00:19Z)
• Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach [53.32564762183639]
  We introduce a novel, unified theoretical framework for watermarking Large Language Models (LLMs)<n>Our approach aims to maximize detection performance while maintaining control over the worst-case false positive rate (FPR) and distortion on text quality.<n>We propose a distortion-free, distribution-adaptive watermarking algorithm (DAWA) that leverages a surrogate model for model-agnosticism and efficiency.
  arXiv  Detail & Related papers  (2024-10-03T18:28:10Z)
• Watermarking Recommender Systems [52.207721219147814]
  We introduce Autoregressive Out-of-distribution Watermarking (AOW), a novel technique tailored specifically for recommender systems. Our approach entails selecting an initial item and querying it through the oracle model, followed by the selection of subsequent items with small prediction scores. To assess the efficacy of the watermark, the model is tasked with predicting the subsequent item given a truncated watermark sequence.
  arXiv  Detail & Related papers  (2024-07-17T06:51:24Z)
• WaterPool: A Watermark Mitigating Trade-offs among Imperceptibility, Efficacy and Robustness [45.27908390001244]
  This paper utilizes a key-centered scheme to unify existing watermarking techniques by decomposing a watermark into two distinct modules. WaterPool is a simple yet effective key module that preserves a complete key sampling space required by imperceptibility while utilizing semantics-based search to improve the key restoration process.
  arXiv  Detail & Related papers  (2024-05-22T10:22:20Z)
• WaterMax: breaking the LLM watermark detectability-robustness-quality trade-off [10.926616117284983]
  Watermarking is a technical means to dissuade malfeasant usage of Large Language Models. This paper proposes a novel watermarking scheme, so-called WaterMax, that enjoys high detectability while sustaining the quality of the generated text.
  arXiv  Detail & Related papers  (2024-03-06T10:55:30Z)
• WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models [48.19623266082828]
  WaterBench is the first comprehensive benchmark for watermarks in large language models (LLMs) We introduce WaterBench, the first comprehensive benchmark for LLM watermarks, in which we design three crucial factors. We evaluate $4$ open-source watermarks on $2$ LLMs under $2$ watermarking strengths and observe the common struggles for current methods on maintaining the generation quality.
  arXiv  Detail & Related papers  (2023-11-13T08:09:01Z)
• Unbiased Watermark for Large Language Models [67.43415395591221]
  This study examines how significantly watermarks impact the quality of model-generated outputs. It is possible to integrate watermarks without affecting the output probability distribution. The presence of watermarks does not compromise the performance of the model in downstream tasks.
  arXiv  Detail & Related papers  (2023-09-22T12:46:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.