Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Models

• URL: http://arxiv.org/abs/2402.14007v2
• Date: Tue, 4 Jun 2024 14:24:15 GMT
• Title: Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Models
• Authors: Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, Rui Wang,
• Abstract summary: We introduce the concept of cross-lingual consistency in text watermarking. Preliminary empirical results reveal that current text watermarking technologies lack consistency when texts are translated into various languages. We propose a Cross-lingual Watermark Removal Attack (CWRA) to bypass watermarking.
• Score: 48.409979469683975
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text watermarking technology aims to tag and identify content produced by large language models (LLMs) to prevent misuse. In this study, we introduce the concept of cross-lingual consistency in text watermarking, which assesses the ability of text watermarks to maintain their effectiveness after being translated into other languages. Preliminary empirical results from two LLMs and three watermarking methods reveal that current text watermarking technologies lack consistency when texts are translated into various languages. Based on this observation, we propose a Cross-lingual Watermark Removal Attack (CWRA) to bypass watermarking by first obtaining a response from an LLM in a pivot language, which is then translated into the target language. CWRA can effectively remove watermarks, decreasing the AUCs to a random-guessing level without performance loss. Furthermore, we analyze two key factors that contribute to the cross-lingual consistency in text watermarking and propose X-SIR as a defense method against CWRA. Code: https://github.com/zwhe99/X-SIR.

Related papers

• Less is More: Sparse Watermarking in LLMs with Enhanced Text Quality [27.592486717044455]
  We present a novel type of watermark, Sparse Watermark, which aims to mitigate this trade-off by applying watermarks to a small subset of generated tokens distributed across the text. Our experimental results demonstrate that the proposed watermarking scheme achieves high detectability while generating text that outperforms previous watermarking methods in quality across various tasks.
  arXiv  Detail & Related papers  (2024-07-17T18:52:12Z)
• On Evaluating The Performance of Watermarked Machine-Generated Texts Under Adversarial Attacks [20.972194348901958]
  We first comb the mainstream watermarking schemes and removal attacks on machine-generated texts. We evaluate eight watermarks (five pre-text, three post-text) and twelve attacks (two pre-text, ten post-text) across 87 scenarios. Results indicate that KGW and Exponential watermarks offer high text quality and watermark retention but remain vulnerable to most attacks.
  arXiv  Detail & Related papers  (2024-07-05T18:09:06Z)
• Mark My Words: Analyzing and Evaluating Language Model Watermarks [8.025719866615333]
  This work focuses on output watermarking techniques, as opposed to image or model watermarks. We focus on three main metrics: quality, size (i.e., the number of tokens needed to detect a watermark), and tamper resistance.
  arXiv  Detail & Related papers  (2023-12-01T01:22:46Z)
• WatME: Towards Lossless Watermarking Through Lexical Redundancy [58.61972059246715]
  This study assesses the impact of watermarking on different capabilities of large language models (LLMs) from a cognitive science lens. We introduce Watermarking with Mutual Exclusion (WatME) to seamlessly integrate watermarks.
  arXiv  Detail & Related papers  (2023-11-16T11:58:31Z)
• Improving the Generation Quality of Watermarked Large Language Models via Word Importance Scoring [81.62249424226084]
  Token-level watermarking inserts watermarks in the generated texts by altering the token probability distributions. This watermarking algorithm alters the logits during generation, which can lead to a downgraded text quality. We propose to improve the quality of texts generated by a watermarked language model by Watermarking with Importance Scoring (WIS)
  arXiv  Detail & Related papers  (2023-11-16T08:36:00Z)
• A Robust Semantics-based Watermark for Large Language Model against Paraphrasing [50.84892876636013]
  Large language models (LLMs) have show great ability in various natural language tasks. There are concerns that LLMs are possible to be used improperly or even illegally. We propose a semantics-based watermark framework SemaMark.
  arXiv  Detail & Related papers  (2023-11-15T06:19:02Z)
• Towards Codable Watermarking for Injecting Multi-bits Information to LLMs [86.86436777626959]
  Large language models (LLMs) generate texts with increasing fluency and realism. Existing watermarking methods are encoding-inefficient and cannot flexibly meet the diverse information encoding needs. We propose Codable Text Watermarking for LLMs (CTWL) that allows text watermarks to carry multi-bit customizable information.
  arXiv  Detail & Related papers  (2023-07-29T14:11:15Z)
• On the Reliability of Watermarks for Large Language Models [95.87476978352659]
  We study the robustness of watermarked text after it is re-written by humans, paraphrased by a non-watermarked LLM, or mixed into a longer hand-written document. We find that watermarks remain detectable even after human and machine paraphrasing. We also consider a range of new detection schemes that are sensitive to short spans of watermarked text embedded inside a large document.
  arXiv  Detail & Related papers  (2023-06-07T17:58:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.