Entropy-Memorization Law: Evaluating Memorization Difficulty of Data in LLMs

• URL: http://arxiv.org/abs/2507.06056v1
• Date: Tue, 08 Jul 2025 14:58:28 GMT
• Title: Entropy-Memorization Law: Evaluating Memorization Difficulty of Data in LLMs
• Authors: Yizhan Huang, Zhe Yang, Meifang Chen, Jianping Zhang, Michael R. Lyu,
• Abstract summary: Large Language Models (LLMs) are known to memorize portions of their training data, sometimes reproducing content verbatim when prompted appropriately.<n>We present the Entropy-Memorization Law, which suggests that data entropy is linearly correlated with memorization score.<n>Adopting the same strategy to discover Entropy-Memorization Law, we derive a simple yet effective approach to distinguish training and testing data.
• Score: 30.55956806927529
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large Language Models (LLMs) are known to memorize portions of their training data, sometimes reproducing content verbatim when prompted appropriately. In this work, we investigate a fundamental yet under-explored question in the domain of memorization: How to characterize memorization difficulty of training data in LLMs? Through empirical experiments on OLMo, a family of open models, we present the Entropy-Memorization Law. It suggests that data entropy is linearly correlated with memorization score. Moreover, in a case study of memorizing highly randomized strings, or "gibberish", we observe that such sequences, despite their apparent randomness, exhibit unexpectedly low empirical entropy compared to the broader training corpus. Adopting the same strategy to discover Entropy-Memorization Law, we derive a simple yet effective approach to distinguish training and testing data, enabling Dataset Inference (DI).

Related papers

• Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis [8.725781605542675]
  Large Language Models (LLMs) achieve remarkable performance through training on massive datasets.<n>LLMs can exhibit concerning behaviors such as verbatim reproduction of training data rather than true generalization.<n>This paper introduces PEARL, a novel approach for detecting memorization in LLMs.
  arXiv  Detail & Related papers  (2025-05-05T20:42:34Z)
• A Geometric Framework for Understanding Memorization in Generative Models [11.263296715798374]
  Recent work has shown that deep generative models can be capable of memorizing and reproducing training datapoints when deployed.<n>These findings call into question the usability of generative models, especially in light of the legal and privacy risks brought about by memorization.<n>We propose the manifold memorization hypothesis (MMH), a geometric framework which leverages the manifold hypothesis into a clear language in which to reason about memorization.
  arXiv  Detail & Related papers  (2024-10-31T18:09:01Z)
• Detecting Training Data of Large Language Models via Expectation Maximization [62.28028046993391]
  We introduce EM-MIA, a novel membership inference method that iteratively refines membership scores and prefix scores via an expectation-maximization algorithm.<n> EM-MIA achieves state-of-the-art results on WikiMIA.
  arXiv  Detail & Related papers  (2024-10-10T03:31:16Z)
• Demystifying Verbatim Memorization in Large Language Models [67.49068128909349]
  Large Language Models (LLMs) frequently memorize long sequences verbatim, often with serious legal and privacy implications. We develop a framework to study verbatim memorization in a controlled setting by continuing pre-training from Pythia checkpoints with injected sequences. We find that (1) non-trivial amounts of repetition are necessary for verbatim memorization to happen; (2) later (and presumably better) checkpoints are more likely to memorize verbatim sequences, even for out-of-distribution sequences.
  arXiv  Detail & Related papers  (2024-07-25T07:10:31Z)
• ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods [56.073335779595475]
  We propose ReCaLL (Relative Conditional Log-Likelihood) to detect pretraining data by leveraging conditional language modeling capabilities.<n>Our empirical findings show that conditioning member data on non-member prefixes induces a larger decrease in log-likelihood compared to non-member data.<n>We conduct comprehensive experiments and show that ReCaLL achieves state-of-the-art performance on the WikiMIA dataset.
  arXiv  Detail & Related papers  (2024-06-23T00:23:13Z)
• Mitigating Approximate Memorization in Language Models via Dissimilarity Learned Policy [0.0]
  Large Language models (LLMs) are trained on large amounts of data. LLMs showed to memorize parts of the training data and emit those data verbatim when an adversary prompts appropriately.
  arXiv  Detail & Related papers  (2023-05-02T15:53:28Z)
• Reconstructing Training Data from Model Gradient, Provably [68.21082086264555]
  We reconstruct the training samples from a single gradient query at a randomly chosen parameter value. As a provable attack that reveals sensitive training data, our findings suggest potential severe threats to privacy.
  arXiv  Detail & Related papers  (2022-12-07T15:32:22Z)
• Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks [12.478605921259403]
  Memorization presents a challenge for several constrained Natural Language Generation (NLG) tasks such as Neural Machine Translation (NMT) We propose a new, inexpensive algorithm for extractive memorization in constrained sequence generation tasks. We develop a simple algorithm which elicits non-memorized translations of memorized samples from the same model.
  arXiv  Detail & Related papers  (2022-10-24T03:01:52Z)
• Quantifying Memorization Across Neural Language Models [61.58529162310382]
  Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized data verbatim. This is undesirable because memorization violates privacy (exposing user data), degrades utility (repeated easy-to-memorize text is often low quality), and hurts fairness (some texts are memorized over others). We describe three log-linear relationships that quantify the degree to which LMs emit memorized training data.
  arXiv  Detail & Related papers  (2022-02-15T18:48:31Z)
• Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation [54.899751055620904]
  We study the phenomenon of memorization with turn-over dropout, an efficient method to estimate influence and memorization, for data with true labels (real data) and data with random labels (random data) Our main findings are: (i) For both real data and random data, the optimization of easy examples (e.g., real data) and difficult examples (e.g., random data) are conducted by the network simultaneously, with easy ones at a higher speed; (ii) For real data, a correct difficult example in the training dataset is more informative than an easy one.
  arXiv  Detail & Related papers  (2021-12-16T11:34:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.