WASA: WAtermark-based Source Attribution for Large Language
Model-Generated Data
- URL: http://arxiv.org/abs/2310.00646v1
- Date: Sun, 1 Oct 2023 12:02:57 GMT
- Title: WASA: WAtermark-based Source Attribution for Large Language
Model-Generated Data
- Authors: Jingtan Wang, Xinyang Lu, Zitong Zhao, Zhongxiang Dai, Chuan-Sheng
Foo, See-Kiong Ng, Bryan Kian Hsiang Low
- Abstract summary: Large language models (LLMs) generate synthetic texts with embedded watermarks that contain information about their source(s)
We propose a WAtermarking for Source Attribution (WASA) framework that satisfies key properties due to our algorithmic designs.
Our framework achieves effective source attribution and data provenance.
- Score: 60.759755177369364
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The impressive performances of large language models (LLMs) and their immense
potential for commercialization have given rise to serious concerns over the
intellectual property (IP) of their training data. In particular, the synthetic
texts generated by LLMs may infringe the IP of the data being used to train the
LLMs. To this end, it is imperative to be able to (a) identify the data
provider who contributed to the generation of a synthetic text by an LLM
(source attribution) and (b) verify whether the text data from a data provider
has been used to train an LLM (data provenance). In this paper, we show that
both problems can be solved by watermarking, i.e., by enabling an LLM to
generate synthetic texts with embedded watermarks that contain information
about their source(s). We identify the key properties of such watermarking
frameworks (e.g., source attribution accuracy, robustness against adversaries),
and propose a WAtermarking for Source Attribution (WASA) framework that
satisfies these key properties due to our algorithmic designs. Our WASA
framework enables an LLM to learn an accurate mapping from the texts of
different data providers to their corresponding unique watermarks, which sets
the foundation for effective source attribution (and hence data provenance).
Extensive empirical evaluations show that our WASA framework achieves effective
source attribution and data provenance.
Related papers
- Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? [62.72729485995075]
We investigate the effectiveness of watermarking as a deterrent against the generation of copyrighted texts.
We find that watermarking adversely affects the success rate of Membership Inference Attacks (MIAs)
We propose an adaptive technique to improve the success rate of a recent MIA under watermarking.
arXiv Detail & Related papers (2024-07-24T16:53:09Z) - Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
Text anonymization is crucial for sharing sensitive data while maintaining privacy.
Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models.
This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
arXiv Detail & Related papers (2024-07-16T14:28:56Z) - SPOT: Text Source Prediction from Originality Score Thresholding [6.790905400046194]
countermeasures aim at detecting misinformation, usually involve domain specific models trained to recognize the relevance of any information.
Instead of evaluating the validity of the information, we propose to investigate LLM generated text from the perspective of trust.
arXiv Detail & Related papers (2024-05-30T21:51:01Z) - Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering [9.86691461253151]
We introduce a novel method for attribution in contextual question answering, leveraging the hidden state representations of large language models (LLMs)
Our approach bypasses the need for extensive model retraining and retrieval model overhead, offering granular attributions and preserving the quality of generated answers.
We present Verifiability-granular, an attribution dataset which has token level annotations for LLM generations in the contextual question answering setup.
arXiv Detail & Related papers (2024-05-28T09:12:44Z) - LLM Attributor: Interactive Visual Attribution for LLM Generation [29.116016627864095]
Python library provides interactive visualizations for training data attribution of large language models.
Our library offers a new way to quickly attribute an LLM's text generation to training data points.
arXiv Detail & Related papers (2024-04-01T13:16:34Z) - Digger: Detecting Copyright Content Mis-usage in Large Language Model
Training [23.99093718956372]
We introduce a framework designed to detect and assess the presence of content from potentially copyrighted books within the training datasets of Large Language Models (LLMs)
This framework also provides a confidence estimation for the likelihood of each content sample's inclusion.
arXiv Detail & Related papers (2024-01-01T06:04:52Z) - DataTales: Investigating the use of Large Language Models for Authoring
Data-Driven Articles [19.341156634212364]
Large language models (LLMs) present an opportunity to assist the authoring of data-driven articles.
We designed a prototype system, DataTales, that leverages a LLM to generate textual narratives accompanying a given chart.
Using DataTales as a design probe, we conducted a qualitative study with 11 professionals to evaluate the concept.
arXiv Detail & Related papers (2023-08-08T06:21:58Z) - Towards Codable Watermarking for Injecting Multi-bits Information to LLMs [86.86436777626959]
Large language models (LLMs) generate texts with increasing fluency and realism.
Existing watermarking methods are encoding-inefficient and cannot flexibly meet the diverse information encoding needs.
We propose Codable Text Watermarking for LLMs (CTWL) that allows text watermarks to carry multi-bit customizable information.
arXiv Detail & Related papers (2023-07-29T14:11:15Z) - Harnessing Explanations: LLM-to-LM Interpreter for Enhanced
Text-Attributed Graph Representation Learning [51.90524745663737]
A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks.
Our method achieves state-of-the-art results on well-established TAG datasets.
Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
arXiv Detail & Related papers (2023-05-31T03:18:03Z) - LLMDet: A Third Party Large Language Models Generated Text Detection
Tool [119.0952092533317]
Large language models (LLMs) are remarkably close to high-quality human-authored text.
Existing detection tools can only differentiate between machine-generated and human-authored text.
We propose LLMDet, a model-specific, secure, efficient, and extendable detection tool.
arXiv Detail & Related papers (2023-05-24T10:45:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.