Optimizing Token Choice for Code Watermarking: An RL Approach

• URL: http://arxiv.org/abs/2508.11925v2
• Date: Sun, 02 Nov 2025 15:47:22 GMT
• Title: Optimizing Token Choice for Code Watermarking: An RL Approach
• Authors: Zhimeng Guo, Huaisheng Zhu, Siyuan Xu, Hangfan Zhang, Teng Xiao, Minhao Cheng,
• Abstract summary: We introduce CodeTracer, an adaptive code watermarking framework underpinned by a novel reinforcement learning paradigm.<n>CodeTracer features a policy-driven approach that utilizes a parameterized model to intelligently bias token choices during next-token prediction.<n>To facilitate policy learning, we devise a comprehensive reward system that seamlessly integrates execution feedback with watermark embedding signals.
• Score: 41.184827829989494
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Protecting intellectual property on LLM-generated code necessitates effective watermarking systems that can operate within code's highly structured, syntactically constrained nature. In this work, we introduce CodeTracer, an innovative adaptive code watermarking framework underpinned by a novel reinforcement learning training paradigm. At its core, CodeTracer features a policy-driven approach that utilizes a parameterized model to intelligently bias token choices during next-token prediction. This strategy ensures that embedded watermarks maintain code functionality while exhibiting subtle yet statistically detectable deviations from typical token distributions. To facilitate policy learning, we devise a comprehensive reward system that seamlessly integrates execution feedback with watermark embedding signals, balancing process-level and outcome-level rewards. Additionally, we employ Gumbel Top-k reparameterization to enable gradient-based optimization of discrete watermarking decisions. Extensive comparative evaluations demonstrate CodeTracer's significant superiority over state-of-the-art baselines in both watermark detectability and the preservation of generated code's functionality.

Related papers

• ALIEN: Analytic Latent Watermarking for Controllable Generation [16.064060838471924]
  We propose an underlineAnaunderlinelytical Watermarkunderlineing Framework for Controllablunderlinee Generatiounderlinen (ALIEN)<n>We develop the first analytical derivation of the time-dependent modulation coefficient that guides the diffusion of watermark residuals to achieve controllable watermark embedding pattern.<n>Results show that ALIEN-Q outperforms the state-of-the-art by 33.1% across 5 quality metrics, and ALIEN-R demonstrates 14.0% improved robustness against generative variant and stability
  arXiv  Detail & Related papers  (2026-02-05T16:04:27Z)
• SWaRL: Safeguard Code Watermarking via Reinforcement Learning [16.888582821315257]
  We present SWaRL, a robust and fidelity-preserving watermarking framework.<n> SWaRL embeds unique and verifiable signatures in the generated output.<n>We show that SWaRL achieves higher watermark detection accuracy compared to prior methods.
  arXiv  Detail & Related papers  (2026-01-05T23:35:39Z)
• CODE ACROSTIC: Robust Watermarking for Code Generation [49.125981508877565]
  Existing methods for watermarking large language models (LLMs) fail to address comment removal attack.<n>Our approach involves leveraging prior knowledge to distinguish between low-entropy and high-entropy parts of the code.<n>We then inject the watermark guided by this Cue List, achieving higher detectability and usability than existing methods.
  arXiv  Detail & Related papers  (2025-12-14T19:14:54Z)
• StableGuard: Towards Unified Copyright Protection and Tamper Localization in Latent Diffusion Models [55.05404953041403]
  We propose a novel framework that seamlessly integrates a binary watermark into the diffusion generation process.<n>We show that StableGuard consistently outperforms state-of-the-art methods in image fidelity, watermark verification, and tampering localization.
  arXiv  Detail & Related papers  (2025-09-22T16:35:19Z)
• Gaussian Shading++: Rethinking the Realistic Deployment Challenge of Performance-Lossless Image Watermark for Diffusion Models [66.54457339638004]
  Copyright protection and inappropriate content generation pose challenges for the practical implementation of diffusion models.<n>We propose a diffusion model watermarking method tailored for real-world deployment.<n>Gaussian Shading++ not only maintains performance losslessness but also outperforms existing methods in terms of robustness.
  arXiv  Detail & Related papers  (2025-04-21T11:18:16Z)
• Robust and Secure Code Watermarking for Large Language Models via ML/Crypto Codesign [15.153228808457628]
  RoSeMary regulates LLM-generated code to avoid intellectual property rights violations and inappropriate misuse in software development.<n>High-quality watermarks adhering to the detectability-fidelity-robustness tri-objective are limited due to codes' low-entropy nature.<n>RoSeMary achieves high detection accuracy while preserving the code functionality.
  arXiv  Detail & Related papers  (2025-02-04T07:35:28Z)
• A Watermark for Order-Agnostic Language Models [55.89285889529492]
  Pattern-mark is a pattern-based watermarking framework specifically designed for order-agnostic LMs. We develop a Markov-chain-based watermark generator that produces watermark key sequences with high-frequency key patterns. Our evaluations on order-agnostic LMs, such as ProteinMPNN and CMLM, demonstrate Pattern-mark's enhanced detection efficiency, generation quality, and robustness.
  arXiv  Detail & Related papers  (2024-10-17T17:41:28Z)
• Theoretically Grounded Framework for LLM Watermarking: A Distribution-Adaptive Approach [53.32564762183639]
  We introduce a novel, unified theoretical framework for watermarking Large Language Models (LLMs)<n>Our approach aims to maximize detection performance while maintaining control over the worst-case false positive rate (FPR) and distortion on text quality.<n>We propose a distortion-free, distribution-adaptive watermarking algorithm (DAWA) that leverages a surrogate model for model-agnosticism and efficiency.
  arXiv  Detail & Related papers  (2024-10-03T18:28:10Z)
• Is The Watermarking Of LLM-Generated Code Robust? [5.48277165801539]
  We show that watermarking techniques are significantly more fragile in code-based contexts.<n>Specifically, we show that simple semantic-preserving transformations, such as variable renaming and dead code insertion, can effectively erase watermarks.
  arXiv  Detail & Related papers  (2024-03-24T21:41:29Z)
• Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models [31.062753031312006]
  Large language models generate high-quality responses with potential misinformation. Watermarking is pivotal in this context, which involves embedding hidden markers in texts. We introduce a novel multi-objective optimization (MOO) approach for watermarking. Our method simultaneously achieves detectability and semantic integrity.
  arXiv  Detail & Related papers  (2024-02-28T05:43:22Z)
• Who Wrote this Code? Watermarking for Code Generation [53.24895162874416]
  We propose Selective WatErmarking via Entropy Thresholding (SWEET) to detect machine-generated text. Our experiments show that SWEET significantly improves code quality preservation while outperforming all baselines.
  arXiv  Detail & Related papers  (2023-05-24T11:49:52Z)
• Towards Tracing Code Provenance with Code Watermarking [37.41260851333952]
  We propose CodeMark, a watermarking system that hides bit strings into variables respecting the natural and operational semantics of the code. For naturalness, we introduce a contextual watermarking scheme to generate watermarked variables more coherent in the context atop graph neural networks. We show CodeMark outperforms the SOTA watermarking systems with a better balance of the watermarking requirements.
  arXiv  Detail & Related papers  (2023-05-21T13:53:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.