Measuring memorization through probabilistic discoverable extraction

• URL: http://arxiv.org/abs/2410.19482v1
• Date: Fri, 25 Oct 2024 11:37:04 GMT
• Title: Measuring memorization through probabilistic discoverable extraction
• Authors: Jamie Hayes, Marika Swanberg, Harsh Chaudhari, Itay Yona, Ilia Shumailov,
• Abstract summary: Large language models (LLMs) are susceptible to memorizing training data. Current methods to measure memorization rates of LLMs rely on single-sequence greedy sampling.
• Score: 19.4511858341881
• License:
• Abstract: Large language models (LLMs) are susceptible to memorizing training data, raising concerns due to the potential extraction of sensitive information. Current methods to measure memorization rates of LLMs, primarily discoverable extraction (Carlini et al., 2022), rely on single-sequence greedy sampling, potentially underestimating the true extent of memorization. This paper introduces a probabilistic relaxation of discoverable extraction that quantifies the probability of extracting a target sequence within a set of generated samples, considering various sampling schemes and multiple attempts. This approach addresses the limitations of reporting memorization rates through discoverable extraction by accounting for the probabilistic nature of LLMs and user interaction patterns. Our experiments demonstrate that this probabilistic measure can reveal cases of higher memorization rates compared to rates found through discoverable extraction. We further investigate the impact of different sampling schemes on extractability, providing a more comprehensive and realistic assessment of LLM memorization and its associated risks. Our contributions include a new probabilistic memorization definition, empirical evidence of its effectiveness, and a thorough evaluation across different models, sizes, sampling schemes, and training data repetitions.

Related papers

• Skewed Memorization in Large Language Models: Quantification and Decomposition [23.097554469918357]
  Memorization in Large Language Models (LLMs) poses privacy and security risks, as models may unintentionally reproduce sensitive or copyrighted data. This paper examines memorization in supervised fine-tuning (SFT), exploring its relationships with training duration, dataset size, and inter-sample similarity.
  arXiv  Detail & Related papers  (2025-02-03T09:23:53Z)
• Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method [108.56493934296687]
  We introduce a divergence-based calibration method, inspired by the divergence-from-randomness concept, to calibrate token probabilities for pretraining data detection. We have developed a Chinese-language benchmark, PatentMIA, to assess the performance of detection approaches for LLMs on Chinese text.
  arXiv  Detail & Related papers  (2024-09-23T07:55:35Z)
• Extracting Training Data from Unconditional Diffusion Models [76.85077961718875]
  diffusion probabilistic models (DPMs) are being employed as mainstream models for generative artificial intelligence (AI) We aim to establish a theoretical understanding of memorization in DPMs with 1) a memorization metric for theoretical analysis, 2) an analysis of conditional memorization with informative and random labels, and 3) two better evaluation metrics for measuring memorization. Based on the theoretical analysis, we propose a novel data extraction method called textbfSurrogate condItional Data Extraction (SIDE) that leverages a trained on generated data as a surrogate condition to extract training data directly from unconditional diffusion models.
  arXiv  Detail & Related papers  (2024-06-18T16:20:12Z)
• To Believe or Not to Believe Your LLM [51.2579827761899]
  We explore uncertainty quantification in large language models (LLMs) We derive an information-theoretic metric that allows to reliably detect when only epistemic uncertainty is large. We conduct a series of experiments which demonstrate the advantage of our formulation.
  arXiv  Detail & Related papers  (2024-06-04T17:58:18Z)
• Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation [73.58618024960968]
  An increasing number of studies are employing large language models (LLMs) as agents to emulate the sequential decision-making processes of humans. This arouses curiosity regarding the capacity of LLM agents to comprehend probability distributions. Our analysis indicates that LLM agents can understand probabilities, but they struggle with probability sampling.
  arXiv  Detail & Related papers  (2024-04-13T16:59:28Z)
• C-ICL: Contrastive In-context Learning for Information Extraction [54.39470114243744]
  c-ICL is a novel few-shot technique that leverages both correct and incorrect sample constructions to create in-context learning demonstrations. Our experiments on various datasets indicate that c-ICL outperforms previous few-shot in-context learning methods.
  arXiv  Detail & Related papers  (2024-02-17T11:28:08Z)
• Querying Easily Flip-flopped Samples for Deep Active Learning [63.62397322172216]
  Active learning is a machine learning paradigm that aims to improve the performance of a model by strategically selecting and querying unlabeled data. One effective selection strategy is to base it on the model's predictive uncertainty, which can be interpreted as a measure of how informative a sample is. This paper proposes the it least disagree metric (LDM) as the smallest probability of disagreement of the predicted label.
  arXiv  Detail & Related papers  (2024-01-18T08:12:23Z)
• TriSig: Assessing the statistical significance of triclusters [2.064612766965483]
  This work proposes a statistical frame to assess the probability of patterns in tensor data to deviate from null expectations. A comprehensive discussion on binomial testing for false positive discoveries is entailed. Results gathered from the application of state-of-the-art triclustering algorithms over distinct real-world case studies in biochemical and biotechnological domains.
  arXiv  Detail & Related papers  (2023-06-01T13:08:54Z)
• Robust Out-of-Distribution Detection on Deep Probabilistic Generative Models [0.06372261626436676]
  Out-of-distribution (OOD) detection is an important task in machine learning systems. Deep probabilistic generative models facilitate OOD detection by estimating the likelihood of a data sample. We propose a new detection metric that operates without outlier exposure.
  arXiv  Detail & Related papers  (2021-06-15T06:36:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.