Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking

• URL: http://arxiv.org/abs/2510.01637v1
• Date: Thu, 02 Oct 2025 03:33:12 GMT
• Title: Detecting Post-generation Edits to Watermarked LLM Outputs via Combinatorial Watermarking
• Authors: Liyan Xie, Muhammad Siddeek, Mohamed Seif, Andrea J. Goldsmith, Mengdi Wang,
• Abstract summary: We introduce a new task: detecting post-generation edits locally made to watermarked LLM outputs.<n>We propose a pattern-based watermarking framework, which partitions the vocabulary into disjoint subsets and embeds the watermark.<n>We evaluate our method on open-source LLMs across a variety of editing scenarios, demonstrating strong empirical performance in edit localization.
• Score: 51.417096446156926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Watermarking has become a key technique for proprietary language models, enabling the distinction between AI-generated and human-written text. However, in many real-world scenarios, LLM-generated content may undergo post-generation edits, such as human revisions or even spoofing attacks, making it critical to detect and localize such modifications. In this work, we introduce a new task: detecting post-generation edits locally made to watermarked LLM outputs. To this end, we propose a combinatorial pattern-based watermarking framework, which partitions the vocabulary into disjoint subsets and embeds the watermark by enforcing a deterministic combinatorial pattern over these subsets during generation. We accompany the combinatorial watermark with a global statistic that can be used to detect the watermark. Furthermore, we design lightweight local statistics to flag and localize potential edits. We introduce two task-specific evaluation metrics, Type-I error rate and detection accuracy, and evaluate our method on open-source LLMs across a variety of editing scenarios, demonstrating strong empirical performance in edit localization.

Related papers

• Yet Another Watermark for Large Language Models [20.295405732813748]
  Existing watermarking methods for large language models (LLMs) embed watermark by adjusting the token sampling prediction or post-processing.<n>We present a new watermarking framework for LLMs, where the watermark is embedded into the LLM by manipulating the internal parameters of the LLM.<n>The proposed method entangles the watermark with the intrinsic parameters of the LLM, which better balances the robustness and imperceptibility of the watermark.
  arXiv  Detail & Related papers  (2025-09-16T02:04:55Z)
• StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models [4.76514657698929]
  StealthInk is a stealthy multi-bit watermarking scheme for large language models (LLMs)<n>It preserves the original text distribution while enabling the embedding of provenance data.<n>We derive a lower bound on the number of tokens necessary for watermark detection at a fixed equal error rate.
  arXiv  Detail & Related papers  (2025-06-05T18:37:38Z)
• In-Context Watermarks for Large Language Models [71.29952527565749]
  In-Context Watermarking (ICW) embeds watermarks into generated text solely through prompt engineering.<n>We investigate four ICW strategies at different levels of granularity, each paired with a tailored detection method.<n>Our experiments validate the feasibility of ICW as a model-agnostic, practical watermarking approach.
  arXiv  Detail & Related papers  (2025-05-22T17:24:51Z)
• Segmenting Watermarked Texts From Language Models [1.4103505579327706]
  This work focuses on a scenario where an untrusted third-party user sends prompts to a trusted language model (LLM) provider, who then generates a text with a watermark. This setup makes it possible for a detector to later identify the source of the text if the user publishes it. We propose a methodology to segment the published text into watermarked and non-watermarked sub-strings.
  arXiv  Detail & Related papers  (2024-10-28T02:05:10Z)
• WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents [63.563031923075066]
  WaterSeeker is a novel approach to efficiently detect and locate watermarked segments amid extensive natural text.<n>It achieves a superior balance between detection accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2024-09-08T14:45:47Z)
• Less is More: Sparse Watermarking in LLMs with Enhanced Text Quality [27.592486717044455]
  We present a novel type of watermark, Sparse Watermark, which aims to mitigate this trade-off by applying watermarks to a small subset of generated tokens distributed across the text. Our experimental results demonstrate that the proposed watermarking scheme achieves high detectability while generating text that outperforms previous watermarking methods in quality across various tasks.
  arXiv  Detail & Related papers  (2024-07-17T18:52:12Z)
• MarkLLM: An Open-Source Toolkit for LLM Watermarking [80.00466284110269]
  MarkLLM is an open-source toolkit for implementing LLM watermarking algorithms. For evaluation, MarkLLM offers a comprehensive suite of 12 tools spanning three perspectives, along with two types of automated evaluation pipelines.
  arXiv  Detail & Related papers  (2024-05-16T12:40:01Z)
• Topic-Based Watermarks for Large Language Models [46.71493672772134]
  We propose a lightweight, topic-guided watermarking scheme for Large Language Model (LLM) output.<n>Our method achieves comparable perplexity to industry-leading systems, including Google's SynthID-Text.
  arXiv  Detail & Related papers  (2024-04-02T17:49:40Z)
• Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models [31.062753031312006]
  Large language models generate high-quality responses with potential misinformation. Watermarking is pivotal in this context, which involves embedding hidden markers in texts. We introduce a novel multi-objective optimization (MOO) approach for watermarking. Our method simultaneously achieves detectability and semantic integrity.
  arXiv  Detail & Related papers  (2024-02-28T05:43:22Z)
• Towards Codable Watermarking for Injecting Multi-bits Information to LLMs [86.86436777626959]
  Large language models (LLMs) generate texts with increasing fluency and realism. Existing watermarking methods are encoding-inefficient and cannot flexibly meet the diverse information encoding needs. We propose Codable Text Watermarking for LLMs (CTWL) that allows text watermarks to carry multi-bit customizable information.
  arXiv  Detail & Related papers  (2023-07-29T14:11:15Z)
• Watermarking Text Generated by Black-Box Language Models [103.52541557216766]
  A watermark-based method was proposed for white-box LLMs, allowing them to embed watermarks during text generation. A detection algorithm aware of the list can identify the watermarked text. We develop a watermarking framework for black-box language model usage scenarios.
  arXiv  Detail & Related papers  (2023-05-14T07:37:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.