Disappearing Ink: Obfuscation Breaks N-gram Code Watermarks in Theory and Practice

• URL: http://arxiv.org/abs/2507.05512v1
• Date: Mon, 07 Jul 2025 22:18:19 GMT
• Title: Disappearing Ink: Obfuscation Breaks N-gram Code Watermarks in Theory and Practice
• Authors: Gehao Zhang, Eugene Bagdasarian, Juan Zhai, Shiqing Ma,
• Abstract summary: Distinguishing AI-generated code from human-written code is crucial for authorship attribution, content tracking, and misuse detection.<n>N-gram-based watermarking schemes have emerged as prominent, which inject secret watermarks to be detected during the generation.<n>Most claims rely solely on defenses against simple code transformations or code optimizations as a simulation of attack, creating a questionable sense of robustness.
• Score: 23.788321123219244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distinguishing AI-generated code from human-written code is becoming crucial for tasks such as authorship attribution, content tracking, and misuse detection. Based on this, N-gram-based watermarking schemes have emerged as prominent, which inject secret watermarks to be detected during the generation. However, their robustness in code content remains insufficiently evaluated. Most claims rely solely on defenses against simple code transformations or code optimizations as a simulation of attack, creating a questionable sense of robustness. In contrast, more sophisticated schemes already exist in the software engineering world, e.g., code obfuscation, which significantly alters code while preserving functionality. Although obfuscation is commonly used to protect intellectual property or evade software scanners, the robustness of code watermarking techniques against such transformations remains largely unexplored. In this work, we formally model the code obfuscation and prove the impossibility of N-gram-based watermarking's robustness with only one intuitive and experimentally verified assumption, distribution consistency, satisfied. Given the original false positive rate of the watermarking detection, the ratio that the detector failed on the watermarked code after obfuscation will increase to 1 - fpr. The experiments have been performed on three SOTA watermarking schemes, two LLMs, two programming languages, four code benchmarks, and four obfuscators. Among them, all watermarking detectors show coin-flipping detection abilities on obfuscated codes (AUROC tightly surrounds 0.5). Among all models, watermarking schemes, and datasets, both programming languages own obfuscators that can achieve attack effects with no detection AUROC higher than 0.6 after the attack. Based on the theoretical and practical observations, we also proposed a potential path of robust code watermarking.

Related papers

• Towards Generalized and Stealthy Watermarking for Generative Code Models [35.78974773421725]
  Experimental results demonstrate that CodeGuard achieves up to 100% watermark verification rates in both code summarization and code generation tasks.<n>In terms of stealthiness, CodeGuard performs exceptionally, with a maximum detection rate of only 0.078 against ONION detection methods.
  arXiv  Detail & Related papers  (2025-06-26T01:14:35Z)
• Decompiling Smart Contracts with a Large Language Model [51.49197239479266]
  Despite Etherscan's 78,047,845 smart contracts deployed on (as of May 26, 2025), a mere 767,520 ( 1%) are open source.<n>This opacity necessitates the automated semantic analysis of on-chain smart contract bytecode.<n>We introduce a pioneering decompilation pipeline that transforms bytecode into human-readable and semantically faithful Solidity code.
  arXiv  Detail & Related papers  (2025-06-24T13:42:59Z)
• Gaussian Shading++: Rethinking the Realistic Deployment Challenge of Performance-Lossless Image Watermark for Diffusion Models [66.54457339638004]
  Copyright protection and inappropriate content generation pose challenges for the practical implementation of diffusion models.<n>We propose a diffusion model watermarking method tailored for real-world deployment.<n>Gaussian Shading++ not only maintains performance losslessness but also outperforms existing methods in terms of robustness.
  arXiv  Detail & Related papers  (2025-04-21T11:18:16Z)
• Robust and Secure Code Watermarking for Large Language Models via ML/Crypto Codesign [15.153228808457628]
  RoSeMary regulates LLM-generated code to avoid intellectual property rights violations and inappropriate misuse in software development.<n>High-quality watermarks adhering to the detectability-fidelity-robustness tri-objective are limited due to codes' low-entropy nature.<n>RoSeMary achieves high detection accuracy while preserving the code functionality.
  arXiv  Detail & Related papers  (2025-02-04T07:35:28Z)
• Edit Distance Robust Watermarks via Indexing Pseudorandom Codes [29.69428894587431]
  Motivated by the problem of detecting AI-generated text, we consider the problem of watermarking the output of language models with provable guarantees.<n>We aim for watermarks which satisfy: (a) undetectability, a cryptographic notion introduced by Christ, Gunn & Zamir (2024) and (b) robustness to channels which introduce a constant fraction of adversarial insertions, substitutions, and deletions.
  arXiv  Detail & Related papers  (2024-06-04T04:03:17Z)
• Is The Watermarking Of LLM-Generated Code Robust? [5.48277165801539]
  We show that watermarking techniques are significantly more fragile in code-based contexts.<n>Specifically, we show that simple semantic-preserving transformations, such as variable renaming and dead code insertion, can effectively erase watermarks.
  arXiv  Detail & Related papers  (2024-03-24T21:41:29Z)
• Zero-Shot Detection of Machine-Generated Codes [83.0342513054389]
  This work proposes a training-free approach for the detection of LLMs-generated codes. We find that existing training-based or zero-shot text detectors are ineffective in detecting code. Our method exhibits robustness against revision attacks and generalizes well to Java codes.
  arXiv  Detail & Related papers  (2023-10-08T10:08:21Z)
• An Unforgeable Publicly Verifiable Watermark for Large Language Models [84.2805275589553]
  Current watermark detection algorithms require the secret key used in the watermark generation process, making them susceptible to security breaches and counterfeiting during public detection. We propose an unforgeable publicly verifiable watermark algorithm named UPV that uses two different neural networks for watermark generation and detection, instead of using the same key at both stages.
  arXiv  Detail & Related papers  (2023-07-30T13:43:27Z)
• Who Wrote this Code? Watermarking for Code Generation [53.24895162874416]
  We propose Selective WatErmarking via Entropy Thresholding (SWEET) to detect machine-generated text. Our experiments show that SWEET significantly improves code quality preservation while outperforming all baselines.
  arXiv  Detail & Related papers  (2023-05-24T11:49:52Z)
• Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication [78.165255859254]
  We propose a reversible watermarking algorithm for integrity authentication. The influence of embedding reversible watermarking on the classification performance is less than 0.5%. At the same time, the integrity of the model can be verified by applying the reversible watermarking.
  arXiv  Detail & Related papers  (2021-04-09T09:32:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.