How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences

• URL: http://arxiv.org/abs/2603.06950v1
• Date: Fri, 06 Mar 2026 23:52:26 GMT
• Title: How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences
• Abstract summary: DNA foundation models have become transformative tools in bioinformatics and healthcare applications.<n>Embeddings-as-a-Service (EBERT) frameworks are increasingly being shared via Embeddings-as-a-Service.<n>This study evaluates the resilience of DNA foundation models to model inversion attacks.
• Score: 0.45880283710344055
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: DNA foundation models have become transformative tools in bioinformatics and healthcare applications. Trained on vast genomic datasets, these models can be used to generate sequence embeddings, dense vector representations that capture complex genomic information. These embeddings are increasingly being shared via Embeddings-as-a-Service (EaaS) frameworks to facilitate downstream tasks, while supposedly protecting the privacy of the underlying raw sequences. However, as this practice becomes more prevalent, the security of these representations is being called into question. This study evaluates the resilience of DNA foundation models to model inversion attacks, whereby adversaries attempt to reconstruct sensitive training data from model outputs. In our study, the model's output for reconstructing the DNA sequence is a zero-shot embedding, which is then fed to a decoder. We evaluated the privacy of three DNA foundation models: DNABERT-2, Evo 2, and Nucleotide Transformer v2 (NTv2). Our results show that per-token embeddings allow near-perfect sequence reconstruction across all models. For mean-pooled embeddings, reconstruction quality degrades as sequence length increases, though it remains substantially above random baselines. Evo 2 and NTv2 prove to be most vulnerable, especially for shorter sequences with reconstruction similarities > 90%, while DNABERT-2's BPE tokenization provides the greatest resilience. We found that the correlation between embedding similarity and sequence similarity was a key predictor of reconstruction success. Our findings emphasize the urgent need for privacy-aware design in genomic foundation models prior to their widespread deployment in EaaS settings. Training code, model weights and evaluation pipeline are released on: https://github.com/not-a-feature/DNA-Embedding-Inversion.

Related papers

• DNA: Dual-stage Native Attribution for Generated Image Source Tracing [75.65470221927096]
  Existing proactive methods rely on watermark embedding or model modification.<n>DNA is a coarse-to-fine framework that follows this hierarchy without additional neural-network training.<n>Experiments show that DNA achieves 89.11% end-to-end attribution accuracy.
  arXiv  Detail & Related papers  (2026-07-15T10:27:52Z)
• DNA Language Models: An Assessment of Pre-Training for Fine-Tuning Tasks [0.0]
  Recent breakthroughs in foundation models and Large Language Models (LLMs) have introduced new opportunities for studying and decoding genomic sequences.<n>Several state-of-the-art approaches, such as DNABERT2, rely on transformer-based architectures, while others, such as ConvNova, still build upon more conventional convolutional models.<n>We investigate three key questions: (i) do transformer-based models provide sufficient improvements upon fine-tuning tasks upon heavy pretraining, (ii) what is the actual contribution of pretraining in this setting, and (iii) how does BPE tokenization impact performance on genomics-related tasks
  arXiv  Detail & Related papers  (2026-06-29T11:20:05Z)
• D3LM: A Discrete DNA Diffusion Language Model for Bidirectional DNA Understanding and Generation [16.814731997144012]
  D3LM (textbfDiscrete textbfDNA textbfDiffusion textbfLanguage textbfModel) achieves bidirectional representation learning and DNA generation through masked diffusion.<n>D3LM directly adopts the Nucleotide Transformer (NT) v2 architecture but reformulates the training objective as masked diffusion in discrete DNA space.
  arXiv  Detail & Related papers  (2026-03-02T12:05:21Z)
• S$^2$Drug: Bridging Protein Sequence and 3D Structure in Contrastive Representation Learning for Virtual Screening [72.89086338778098]
  We propose a two-stage framework for protein-ligand contrastive representation learning.<n>In the first stage, we perform protein sequence pretraining on ChemBL using an ESM2-based backbone.<n>In the second stage, we fine-tune on PDBBind by fusing sequence and structure information through a residue-level gating module.<n>This auxiliary task guides the model to accurately localize binding residues within the protein sequence and capture their 3D spatial arrangement.
  arXiv  Detail & Related papers  (2025-11-10T11:57:47Z)
• Comparing Reconstruction Attacks on Pretrained Versus Full Fine-tuned Large Language Model Embeddings on Homo Sapiens Splice Sites Genomic Data [4.729128449905504]
  This study investigates embedding reconstruction attacks in large language models applied to genomic sequences.<n>We conduct a comprehensive analysis using the HS3D genomic dataset to determine whether task-specific optimization strengthens or weakens privacy protections.
  arXiv  Detail & Related papers  (2025-11-09T15:38:35Z)
• eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence Analysis [5.86106644437914]
  Extrachromosomal circular DNA (eccDNA) plays key regulatory roles and contributes to oncogene overexpression in cancer.<n>No pre-trained models currently support full-length circular eccDNA for downstream analysis.<n>eccDNAMamba is the first bidirectional state-space encoder tailored for circular DNA sequences.
  arXiv  Detail & Related papers  (2025-06-22T17:50:57Z)
• Regulatory DNA sequence Design with Reinforcement Learning [56.20290878358356]
  We propose a generative approach that leverages reinforcement learning to fine-tune a pre-trained autoregressive model.<n>We evaluate our method on promoter design tasks in two yeast media conditions and enhancer design tasks for three human cell types.
  arXiv  Detail & Related papers  (2025-03-11T02:33:33Z)
• DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA Embeddings [7.822348354050447]
  We introduce DNABERT-S, a tailored genome model that develops species-aware embeddings to naturally cluster and segregate DNA sequences of different species. Emerged results on 23 diverse datasets show DNABERT-S's effectiveness, especially in realistic label-scarce scenarios.
  arXiv  Detail & Related papers  (2024-02-13T20:21:29Z)
• DNAGPT: A Generalized Pre-trained Tool for Versatile DNA Sequence Analysis Tasks [14.931476374660944]
  DNAGPT is a generalized DNA pre-training model trained on over 200 billion base pairs from all mammals. By enhancing the classic GPT model with a binary classification task, a numerical regression task, and a comprehensive token language, DNAGPT can handle versatile DNA analysis tasks.
  arXiv  Detail & Related papers  (2023-07-11T06:30:43Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality [84.94877848357896]
  Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models. We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples. We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
  arXiv  Detail & Related papers  (2022-11-28T17:36:41Z)
• Reprogramming Pretrained Language Models for Antibody Sequence Infilling [72.13295049594585]
  Computational design of antibodies involves generating novel and diverse sequences, while maintaining structural consistency. Recent deep learning models have shown impressive results, however the limited number of known antibody sequence/structure pairs frequently leads to degraded performance. In our work we address this challenge by leveraging Model Reprogramming (MR), which repurposes pretrained models on a source language to adapt to the tasks that are in a different language and have scarce data.
  arXiv  Detail & Related papers  (2022-10-05T20:44:55Z)
• DNA-GCN: Graph convolutional networks for predicting DNA-protein binding [4.1600531290054]
  We build a sequence k-mer graph and learn DNA Graph Convolutional Network (DNA-GCN) for the whole dataset. DNA-GCN is with a one-hot representation for all nodes, and it then jointly learns the embeddings for both k-mers and sequences. We evaluate our model on 50 datasets from ENCODE.
  arXiv  Detail & Related papers  (2021-06-02T07:36:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.