WoE Wrote It? Watermarking Mixture-of-Experts LLMs for Black-Box Text Provenance

• URL: http://arxiv.org/abs/2608.29151v1
• Date: Sat, 29 Aug 2026 08:57:06 GMT
• Title: WoE Wrote It? Watermarking Mixture-of-Experts LLMs for Black-Box Text Provenance
• Abstract summary: We introduce Watermarking of Experts (WoE), a novel black-box text provenance method.<n>WoE biases the vocabulary of specific experts and shifts the watermark signal away from unenforceable inference wrappers.<n>We evaluate WoE across eight MoE models, demonstrating successful watermark detection from suspect text.
• Score: 14.891975420982504
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Model (LLM) watermarks provide a mechanism for text provenance, enabling model owners to identify machine-generated content and attribute it to a specific watermarked model. However, current LLM watermarking approaches predominantly rely on inference-time sampler methods and focus their analysis on dense models. Inference-time methods are only effective when the text is explicitly generated via the model owner's controlled API; they fail in a post-compromise scenario. An adversary who steals or leaks the model weights gains complete control over inference and can simply run an unmodified sampler, bypassing the watermark and preventing post-theft attribution. In this work, we introduce Watermarking of Experts (WoE), a novel black-box text provenance method that leverages the unique structural properties of sparse Mixture-of-Experts (MoE) models. WoE biases the vocabulary of specific experts and shifts the watermark signal embedding away from unenforceable inference wrappers. This approach ensures the watermark remains intrinsic to the model parameters, enabling defenders to attribute text generated by stolen weights, leaked checkpoints, and secondary dense models distilled from the stolen architecture without needing access to the adversary's deployment or weights. We evaluate WoE across eight MoE models, demonstrating successful watermark detection from suspect text, achieving an average true positive rate of 90.1% at a 1% false positive rate, reaching up to 94.9%, while largely preserving general model utility. Furthermore, WoE remains detectable under adversarial supervised fine-tuning, model extraction, and output-level paraphrasing, forcing malicious actors into a trade-off in which weakening the attribution signal requires additional model adaptation or text-rewriting operations, or compromises the utility of the resulting output.

Related papers

• DeepTracer: Tracing Stolen Model via Deep Coupled Watermarks [14.552367035706283]
  We introduce a robust watermarking framework, DeepTracer, which leverages a novel watermark samples construction method and a same-class coupling loss constraint.<n>DeepTracer can incur a high-coupling model between watermark task and primary task that makes adversaries learn the hidden watermark task when stealing the primary task functionality.<n>We propose an effective watermark samples filtering mechanism that elaborately select watermark key samples used in model ownership verification to enhance the reliability of watermarks.
  arXiv  Detail & Related papers  (2025-11-12T05:06:25Z)
• LLM Watermark Evasion via Bias Inversion [24.543675977310357]
  We propose the emphBias-Inversion Rewriting Attack (BIRA), which is theoretically motivated and model-agnostic.<n>BIRA weakens the watermark signal by suppressing the logits of likely watermarked tokens during rewriting, without any knowledge of the underlying watermarking scheme.
  arXiv  Detail & Related papers  (2025-09-27T00:24:57Z)
• Character-Level Perturbations Disrupt LLM Watermarks [64.60090923837701]
  We formalize the system model for Large Language Model (LLM) watermarking.<n>We characterize two realistic threat models constrained on limited access to the watermark detector.<n>We demonstrate character-level perturbations are significantly more effective for watermark removal under the most restrictive threat model.<n> Experiments confirm the superiority of character-level perturbations and the effectiveness of the Genetic Algorithm (GA) in removing watermarks under realistic constraints.
  arXiv  Detail & Related papers  (2025-09-11T02:50:07Z)
• Towards Dataset Copyright Evasion Attack against Personalized Text-to-Image Diffusion Models [52.877452505561706]
  We propose the first copyright evasion attack specifically designed to undermine dataset ownership verification (DOV)<n>Our CEAT2I comprises three stages: watermarked sample detection, trigger identification, and efficient watermark mitigation.<n>Our experiments show that our CEAT2I effectively evades DOV mechanisms while preserving model performance.
  arXiv  Detail & Related papers  (2025-05-05T17:51:55Z)
• AGATE: Stealthy Black-box Watermarking for Multimodal Model Copyright Protection [26.066755429896926]
  Methods select Out-of-Distribution (OoD) data as backdoor watermarks and retrain the original model for copyright protection.<n>Existing methods are susceptible to malicious detection and forgery by adversaries, resulting in watermark evasion.<n>We propose Model-underlineagnostic Black-box Backdoor Wunderlineatermarking Framework (AGATE) to address stealthiness and robustness challenges in multimodal model copyright protection.
  arXiv  Detail & Related papers  (2025-04-28T14:52:01Z)
• GaussMark: A Practical Approach for Structural Watermarking of Language Models [61.84270985214254]
  GaussMark is a simple, efficient, and relatively robust scheme for watermarking large language models.<n>We show that GaussMark is reliable, efficient, and relatively robust to corruptions such as insertions, deletions, substitutions, and roundtrip translations.
  arXiv  Detail & Related papers  (2025-01-17T22:30:08Z)
• AquaLoRA: Toward White-box Protection for Customized Stable Diffusion Models via Watermark LoRA [67.68750063537482]
  Diffusion models have achieved remarkable success in generating high-quality images. Recent works aim to let SD models output watermarked content for post-hoc forensics. We propose textttmethod as the first implementation under this scenario.
  arXiv  Detail & Related papers  (2024-05-18T01:25:47Z)
• Watermarking Language Models for Many Adaptive Users [47.90822587139056]
  We study watermarking schemes for language models with provable guarantees. We introduce multi-user watermarks, which allow tracing model-generated text to individual users. We prove that the undetectable zero-bit scheme of Christ, Gunn, and Zamir (2024) is adaptively robust.
  arXiv  Detail & Related papers  (2024-05-17T22:15:30Z)
• Explanation as a Watermark: Towards Harmless and Multi-bit Model Ownership Verification via Watermarking Feature Attribution [22.933101948176606]
  backdoor-based model watermarks are the primary and cutting-edge methods to implant such properties in released models. We design a new watermarking paradigm, $i.e.$, Explanation as a Watermark (EaaW), that implants verification behaviors into the explanation of feature attribution.
  arXiv  Detail & Related papers  (2024-05-08T05:49:46Z)
• ModelShield: Adaptive and Robust Watermark against Model Extraction Attack [58.46326901858431]
  Large language models (LLMs) demonstrate general intelligence across a variety of machine learning tasks.<n> adversaries can still utilize model extraction attacks to steal the model intelligence encoded in model generation.<n> Watermarking technology offers a promising solution for defending against such attacks by embedding unique identifiers into the model-generated content.
  arXiv  Detail & Related papers  (2024-05-03T06:41:48Z)
• ClearMark: Intuitive and Robust Model Watermarking via Transposed Model Training [50.77001916246691]
  This paper introduces ClearMark, the first DNN watermarking method designed for intuitive human assessment. ClearMark embeds visible watermarks, enabling human decision-making without rigid value thresholds. It shows an 8,544-bit watermark capacity comparable to the strongest existing work.
  arXiv  Detail & Related papers  (2023-10-25T08:16:55Z)
• Towards Robust Model Watermark via Reducing Parametric Vulnerability [57.66709830576457]
  backdoor-based ownership verification becomes popular recently, in which the model owner can watermark the model. We propose a mini-max formulation to find these watermark-removed models and recover their watermark behavior. Our method improves the robustness of the model watermarking against parametric changes and numerous watermark-removal attacks.
  arXiv  Detail & Related papers  (2023-09-09T12:46:08Z)
• Entangled Watermarks as a Defense against Model Extraction [42.74645868767025]
  Entangled Watermarking Embeddings (EWE) are used to protect machine learning models fromExtraction attacks. EWE learns features for classifying data that is sampled from the task distribution and data that encodes watermarks. Experiments on MNIST, Fashion-MNIST, CIFAR-10, and Speech Commands validate that the defender can claim model ownership with 95% confidence with less than 100 queries to the stolen copy.
  arXiv  Detail & Related papers  (2020-02-27T15:47:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.