Rethinking LLM Memorization through the Lens of Adversarial Compression

• URL: http://arxiv.org/abs/2404.15146v2
• Date: Mon, 1 Jul 2024 14:43:11 GMT
• Title: Rethinking LLM Memorization through the Lens of Adversarial Compression
• Authors: Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zachary C. Lipton, J. Zico Kolter,
• Abstract summary: Large language models (LLMs) trained on web-scale datasets raise substantial concerns regarding permissible data usage. One major question is whether these models "memorize" all their training data or they integrate many data sources in some way more akin to how a human would learn and synthesize information. We propose the Adversarial Compression Ratio (ACR) as a metric for assessing memorization in LLMs.
• Score: 93.13830893086681
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) trained on web-scale datasets raise substantial concerns regarding permissible data usage. One major question is whether these models "memorize" all their training data or they integrate many data sources in some way more akin to how a human would learn and synthesize information. The answer hinges, to a large degree, on how we define memorization. In this work, we propose the Adversarial Compression Ratio (ACR) as a metric for assessing memorization in LLMs. A given string from the training data is considered memorized if it can be elicited by a prompt (much) shorter than the string itself -- in other words, if these strings can be "compressed" with the model by computing adversarial prompts of fewer tokens. The ACR overcomes the limitations of existing notions of memorization by (i) offering an adversarial view of measuring memorization, especially for monitoring unlearning and compliance; and (ii) allowing for the flexibility to measure memorization for arbitrary strings at a reasonably low compute. Our definition serves as a practical tool for determining when model owners may be violating terms around data usage, providing a potential legal tool and a critical lens through which to address such scenarios.

Related papers

• Unlocking Memorization in Large Language Models with Dynamic Soft Prompting [66.54460367290146]
  Large language models (LLMs) have revolutionized natural language processing (NLP) tasks such as summarization, question answering, and translation. LLMs pose significant security risks due to their tendency to memorize training data, leading to potential privacy breaches and copyright infringement. We propose a novel method for estimating LLM memorization using dynamic, prefix-dependent soft prompts.
  arXiv  Detail & Related papers  (2024-09-20T18:56:32Z)
• Soft Prompting for Unlearning in Large Language Models [11.504012974208466]
  This work focuses on investigating machine unlearning for Large Language Models motivated by data protection regulations. We propose a framework textbfSoft textbfPrompting for textbfUntextbflearning (SPUL) We conduct a rigorous evaluation of the proposed method and our results indicate that SPUL can significantly improve the trade-off between utility and forgetting.
  arXiv  Detail & Related papers  (2024-06-17T19:11:40Z)
• SoK: Memorization in General-Purpose Large Language Models [25.448127387943053]
  Large Language Models (LLMs) are advancing at a remarkable pace, with myriad applications under development. LLMs can memorize short secrets in the training data, but can also memorize concepts like facts or writing styles that can be expressed in text in many different ways. We propose a taxonomy for memorization in LLMs that covers verbatim text, facts, ideas and algorithms, writing styles, distributional properties, and alignment goals.
  arXiv  Detail & Related papers  (2023-10-24T14:25:53Z)
• Quantifying and Analyzing Entity-level Memorization in Large Language Models [4.59914731734176]
  Large language models (LLMs) have been proven capable of memorizing their training data. Privacy risks arising from memorization have attracted increasing attention. We propose a fine-grained, entity-level definition to quantify memorization with conditions and metrics closer to real-world scenarios.
  arXiv  Detail & Related papers  (2023-08-30T03:06:47Z)
• Unveiling Memorization in Code Models [13.867618700182486]
  A code model memorizes and produces source code verbatim, which potentially contains vulnerabilities, sensitive information, or code with strict licenses. This paper investigates what extent do code models memorize their training data? We build a taxonomy of memorized contents with 3 categories and 14 subcategories.
  arXiv  Detail & Related papers  (2023-08-19T07:25:39Z)
• In-context Autoencoder for Context Compression in a Large Language Model [70.7621953091318]
  We propose the In-context Autoencoder (ICAE) to compress a long context into short compact memory slots. ICAE is first pretrained using both autoencoding and language modeling objectives on massive text data.
  arXiv  Detail & Related papers  (2023-07-13T17:59:21Z)
• Mitigating Approximate Memorization in Language Models via Dissimilarity Learned Policy [0.0]
  Large Language models (LLMs) are trained on large amounts of data. LLMs showed to memorize parts of the training data and emit those data verbatim when an adversary prompts appropriately.
  arXiv  Detail & Related papers  (2023-05-02T15:53:28Z)
• Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy [91.98116450958331]
  We argue that verbatim memorization definitions are too restrictive and fail to capture more subtle forms of memorization. Specifically, we design and implement an efficient defense that perfectly prevents all verbatim memorization. We conclude by discussing potential alternative definitions and why defining memorization is a difficult yet crucial open question for neural language models.
  arXiv  Detail & Related papers  (2022-10-31T17:57:55Z)
• Towards Differential Relational Privacy and its use in Question Answering [109.4452196071872]
  Memorization of relation between entities in a dataset can lead to privacy issues when using a trained question answering model. We quantify this phenomenon and provide a possible definition of Differential Privacy (DPRP) We illustrate concepts in experiments with largescale models for Question Answering.
  arXiv  Detail & Related papers  (2022-03-30T22:59:24Z)
• Quantifying Memorization Across Neural Language Models [61.58529162310382]
  Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized data verbatim. This is undesirable because memorization violates privacy (exposing user data), degrades utility (repeated easy-to-memorize text is often low quality), and hurts fairness (some texts are memorized over others). We describe three log-linear relationships that quantify the degree to which LMs emit memorized training data.
  arXiv  Detail & Related papers  (2022-02-15T18:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.