Tracing Text Provenance via Context-Aware Lexical Substitution

• URL: http://arxiv.org/abs/2112.07873v1
• Date: Wed, 15 Dec 2021 04:27:33 GMT
• Title: Tracing Text Provenance via Context-Aware Lexical Substitution
• Authors: Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, Nenghai Yu
• Abstract summary: We propose a natural language watermarking scheme based on context-aware lexical substitution. Under both objective and subjective metrics, our watermarking scheme can well preserve the semantic integrity of original sentences.
• Score: 81.49359106648735
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text content created by humans or language models is often stolen or misused by adversaries. Tracing text provenance can help claim the ownership of text content or identify the malicious users who distribute misleading content like machine-generated fake news. There have been some attempts to achieve this, mainly based on watermarking techniques. Specifically, traditional text watermarking methods embed watermarks by slightly altering text format like line spacing and font, which, however, are fragile to cross-media transmissions like OCR. Considering this, natural language watermarking methods represent watermarks by replacing words in original sentences with synonyms from handcrafted lexical resources (e.g., WordNet), but they do not consider the substitution's impact on the overall sentence's meaning. Recently, a transformer-based network was proposed to embed watermarks by modifying the unobtrusive words (e.g., function words), which also impair the sentence's logical and semantic coherence. Besides, one well-trained network fails on other different types of text content. To address the limitations mentioned above, we propose a natural language watermarking scheme based on context-aware lexical substitution (LS). Specifically, we employ BERT to suggest LS candidates by inferring the semantic relatedness between the candidates and the original sentence. Based on this, a selection strategy in terms of synchronicity and substitutability is further designed to test whether a word is exactly suitable for carrying the watermark signal. Extensive experiments demonstrate that, under both objective and subjective metrics, our watermarking scheme can well preserve the semantic integrity of original sentences and has a better transferability than existing methods. Besides, the proposed LS approach outperforms the state-of-the-art approach on the Stanford Word Substitution Benchmark.

Related papers

• Can Watermarks Survive Translation? On the Cross-lingual Consistency of Text Watermark for Large Language Models [48.409979469683975]
  We introduce the concept of cross-lingual consistency in text watermarking. Preliminary empirical results reveal that current text watermarking technologies lack consistency when texts are translated into various languages. We propose a Cross-lingual Watermark Removal Attack (CWRA) to bypass watermarking.
  arXiv  Detail & Related papers  (2024-02-21T18:48:38Z)
• Provably Robust Multi-bit Watermarking for AI-generated Text [37.21416140194606]
  Large Language Models (LLMs) have demonstrated remarkable capabilities of generating texts resembling human language. They can be misused by criminals to create deceptive content, such as fake news and phishing emails. Watermarking is a key technique to address these concerns, which embeds a message into a text.
  arXiv  Detail & Related papers  (2024-01-30T08:46:48Z)
• WatME: Towards Lossless Watermarking Through Lexical Redundancy [58.61972059246715]
  This study assesses the impact of watermarking on different capabilities of large language models (LLMs) from a cognitive science lens. We introduce Watermarking with Mutual Exclusion (WatME) to seamlessly integrate watermarks.
  arXiv  Detail & Related papers  (2023-11-16T11:58:31Z)
• A Robust Semantics-based Watermark for Large Language Model against Paraphrasing [50.84892876636013]
  Large language models (LLMs) have show great ability in various natural language tasks. There are concerns that LLMs are possible to be used improperly or even illegally. We propose a semantics-based watermark framework SemaMark.
  arXiv  Detail & Related papers  (2023-11-15T06:19:02Z)
• SemStamp: A Semantic Watermark with Paraphrastic Robustness for Text Generation [72.10931780019297]
  Existing watermarking algorithms are vulnerable to paraphrase attacks because of their token-level design. We propose SemStamp, a robust sentence-level semantic watermarking algorithm based on locality-sensitive hashing (LSH) Experimental results show that our novel semantic watermark algorithm is not only more robust than the previous state-of-the-art method on both common and bigram paraphrase attacks, but also is better at preserving the quality of generation.
  arXiv  Detail & Related papers  (2023-10-06T03:33:42Z)
• Towards Codable Watermarking for Injecting Multi-bits Information to LLMs [86.86436777626959]
  Large language models (LLMs) generate texts with increasing fluency and realism. Existing watermarking methods are encoding-inefficient and cannot flexibly meet the diverse information encoding needs. We propose Codable Text Watermarking for LLMs (CTWL) that allows text watermarks to carry multi-bit customizable information.
  arXiv  Detail & Related papers  (2023-07-29T14:11:15Z)
• Watermarking Text Generated by Black-Box Language Models [103.52541557216766]
  A watermark-based method was proposed for white-box LLMs, allowing them to embed watermarks during text generation. A detection algorithm aware of the list can identify the watermarked text. We develop a watermarking framework for black-box language model usage scenarios.
  arXiv  Detail & Related papers  (2023-05-14T07:37:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.