Quantifying Memorization and Privacy Risks in Genomic Language Models

• URL: http://arxiv.org/abs/2603.08913v1
• Date: Mon, 09 Mar 2026 20:30:37 GMT
• Title: Quantifying Memorization and Privacy Risks in Genomic Language Models
• Abstract summary: Genomic language models (GLMs) have emerged as powerful tools for learning representations of DNA sequences.<n>GLMs risk memorizing specific sequences from their training data, raising serious concerns around privacy, data leakage, and regulatory compliance.<n>We present a comprehensive, multi-vector privacy evaluation framework designed to quantify memorization risks in GLMs.
• Score: 46.592953963976356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Genomic language models (GLMs) have emerged as powerful tools for learning representations of DNA sequences, enabling advances in variant prediction, regulatory element identification, and cross-task transfer learning. However, as these models are increasingly trained or fine-tuned on sensitive genomic cohorts, they risk memorizing specific sequences from their training data, raising serious concerns around privacy, data leakage, and regulatory compliance. Despite growing awareness of memorization risks in general-purpose language models, little systematic evaluation exists for these risks in the genomic domain, where data exhibit unique properties such as a fixed nucleotide alphabet, strong biological structure, and individual identifiability. We present a comprehensive, multi-vector privacy evaluation framework designed to quantify memorization risks in GLMs. Our approach integrates three complementary risk assessment methodologies: perplexity-based detection, canary sequence extraction, and membership inference. These are combined into a unified evaluation pipeline that produces a worst-case memorization risk score. To enable controlled evaluation, we plant canary sequences at varying repetition rates into both synthetic and real genomic datasets, allowing precise quantification of how repetition and training dynamics influence memorization. We evaluate our framework across multiple GLM architectures, examining the relationship between sequence repetition, model capacity, and memorization risk. Our results establish that GLMs exhibit measurable memorization and that the degree of memorization varies across architectures and training regimes. These findings reveal that no single attack vector captures the full scope of memorization risk, underscoring the need for multi-vector privacy auditing as a standard practice for genomic AI systems.

Related papers

• On the Privacy of LLMs: An Ablation Study [0.0]
  Large language models (LLMs) are increasingly deployed in interactive and retrieval-augmented settings.<n>We introduce a unified threat model and notation, reproduce a representative set of privacy attacks, and conduct a structured ablation study.
  arXiv  Detail & Related papers  (2026-05-04T06:06:41Z)
• Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation [50.87199039334856]
  Retrieval-Augmented Generation (RAG) has become a cornerstone of knowledge-intensive applications.<n>Recent studies show that knowledge-extraction attacks can recover sensitive knowledge-base content through maliciously crafted queries.<n>We introduce the first systematic benchmark for knowledge-extraction attacks on RAG systems.
  arXiv  Detail & Related papers  (2026-02-10T01:27:46Z)
• The Hidden Cost of Modeling P(X): Vulnerability to Membership Inference Attacks in Generative Text Classifiers [6.542294761666199]
  Membership Inference Attacks (MIAs) pose a critical privacy threat by enabling adversaries to determine whether a specific sample was included in a model's training dataset.<n>We show that fully generative classifiers which explicitly model the joint likelihood $P(X,Y)$ are most vulnerable to membership leakage.
  arXiv  Detail & Related papers  (2025-10-17T18:09:33Z)
• Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning [50.45435841411193]
  Code Language Models (CLMs) exhibit unintended memorization of sensitive training data, enabling verbatim reproduction of confidential information when specifically prompted.<n>We introduce CodeEraser, an advanced variant that selectively unlearns sensitive memorized segments in code while preserving the structural integrity and functional correctness of the surrounding code.
  arXiv  Detail & Related papers  (2025-09-17T07:12:35Z)
• Memorization in Language Models through the Lens of Intrinsic Dimension [0.0]
  Language Models (LMs) are prone to memorizing parts of their data during training and unintentionally emitting them at generation time.<n>We investigate the role of Intrinsic Dimension (ID), a geometric proxy for the structural complexity of a sequence in latent space, in modulating memorization.
  arXiv  Detail & Related papers  (2025-06-11T10:42:27Z)
• Skewed Memorization in Large Language Models: Quantification and Decomposition [23.097554469918357]
  Memorization in Large Language Models (LLMs) poses privacy and security risks, as models may unintentionally reproduce sensitive or copyrighted data.<n>This paper examines memorization in supervised fine-tuning (SFT), exploring its relationships with training duration, dataset size, and inter-sample similarity.
  arXiv  Detail & Related papers  (2025-02-03T09:23:53Z)
• Detecting Memorization in Large Language Models [0.0]
  Large language models (LLMs) have achieved impressive results in natural language processing but are prone to memorizing portions of their training data.<n>Traditional methods for detecting memorization rely on output probabilities or loss functions.<n>We introduce an analytical method that precisely detects memorization by examining neuron activations within the LLM.
  arXiv  Detail & Related papers  (2024-12-02T00:17:43Z)
• Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models [7.50189359952191]
  We show that sequences which are not memorized after the first encounter can be "uncovered" throughout the course of training. The presence of latent memorization presents a challenge for data privacy as memorized sequences may be hidden at the final checkpoint of the model. We develop a diagnostic test relying on the cross entropy loss to uncover latent memorized sequences with high accuracy.
  arXiv  Detail & Related papers  (2024-06-20T17:56:17Z)
• Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks [142.67349734180445]
  Existing algorithms that provide risk-awareness to deep neural networks are complex and ad-hoc. Here we present capsa, a framework for extending models with risk-awareness.
  arXiv  Detail & Related papers  (2023-08-01T02:07:47Z)
• Exploring Memorization in Adversarial Training [58.38336773082818]
  We investigate the memorization effect in adversarial training (AT) for promoting a deeper understanding of capacity, convergence, generalization, and especially robust overfitting. We propose a new mitigation algorithm motivated by detailed memorization analyses.
  arXiv  Detail & Related papers  (2021-06-03T05:39:57Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.