EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors

• URL: http://arxiv.org/abs/2602.21218v1
• Date: Sat, 31 Jan 2026 00:04:08 GMT
• Title: EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors
• Abstract summary: We introduce EPSVec, a lightweight alternative to existing private text generation methods.<n> EPSVec steers generation using *dataset vectors*--directions in activation space that capture the distributional gap between private data and public priors.<n>Our experiments demonstrate that EPSVec outperforms existing baselines in distributional alignment and downstream utility.
• Score: 28.30281053018808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-quality data is essential for modern machine learning, yet many valuable corpora are sensitive and cannot be freely shared. Synthetic data offers a practical substitute for downstream development, and large language models (LLMs) have emerged as powerful engines for generating it. However, existing private text generation methods are severely inefficient: they are data-intensive, computationally slow, and often require large private corpora or batch sizes to achieve usable quality. We introduce EPSVec, a differentially-private lightweight alternative that steers LLM generation using *dataset vectors*--directions in activation space that capture the distributional gap between private data and public priors. EPSVec extracts and sanitizes steering vectors just once and then performs standard decoding. This decouples the privacy budget from generation, enabling arbitrarily many synthetic samples without additional privacy cost and yielding strong fidelity even in low-data regimes. Furthermore, we enhance our method by utilizing pretrained (base) models and introducing fixed-shot prompting to boost generation diversity and fidelity. Our experiments demonstrate that EPSVec outperforms existing baselines in distributional alignment and downstream utility, particularly in low-data regimes, while significantly reducing computational overhead.

Related papers

• MAPLE: Metadata Augmented Private Language Evolution [31.885554778844867]
  We propose Metadata Augmented Private Language Evolution (MAPLE) to ground the initial synthetic distribution in the target domain.<n> MAPLE achieves a significantly more favorable privacy-utility trade-off, converges faster, and drastically reduces API costs compared to previous PE methods.
  arXiv  Detail & Related papers  (2026-02-26T14:18:36Z)
• Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning [49.04912820721943]
  Supervised fine-tuning (SFT) is computationally expensive and sometimes suffers from overfitting or bias amplification.<n>This work studies the online batch selection family that dynamically scores and filters samples during the training process.<n>We develop textbfUDS (Utility-Diversity Sampling), a framework for efficient online batch selection in SFT.
  arXiv  Detail & Related papers  (2025-10-19T15:32:01Z)
• Learning More with Less: A Generalizable, Self-Supervised Framework for Privacy-Preserving Capacity Estimation with EV Charging Data [84.37348569981307]
  We propose a first-of-its-kind capacity estimation model based on self-supervised pre-training.<n>Our model consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-10-05T08:58:35Z)
• Synthetic Data Generation and Differential Privacy using Tensor Networks' Matrix Product States (MPS) [33.032422801043495]
  We propose a method for generating privacy-preserving high-quality synthetic data using Matrix Product States (MPS)<n>We benchmark the MPS-based generative model against state-of-the-art models such as CTGAN, VAE, and PrivBayes.<n>Our results show that MPS outperforms classical models, particularly under strict privacy constraints.
  arXiv  Detail & Related papers  (2025-08-08T12:14:57Z)
• Improving Noise Efficiency in Privacy-preserving Dataset Distillation [59.57846442477106]
  We introduce a novel framework that decouples sampling from optimization for better convergence and improves signal quality.<n>On CIFAR-10, our method achieves a textbf10.0% improvement with 50 images per class and textbf8.3% increase with just textbfone-fifth the distilled set size of previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-08-03T13:15:52Z)
• A Distributed Generative AI Approach for Heterogeneous Multi-Domain Environments under Data Sharing constraints [0.6943041855623096]
  We propose a novel approach for decentralized GAN training.<n>It enables the utilization of distributed data and underutilized, low-capability devices while not sharing data in its raw form.<n>Our approach is designed to tackle key challenges in decentralized environments.
  arXiv  Detail & Related papers  (2025-07-17T10:31:31Z)
• Embedding-Based Federated Data Sharing via Differentially Private Conditional VAEs [0.13108652488669734]
  Federated Learning (FL) enables decentralized training but suffers from high communication costs.<n>We propose a data-sharing method via Differentially Private (DP) generative models.<n>Clients collaboratively train a Differentially Private Conditional Variational Autoencoder (DP-CVAE) to model a global, privacy-aware data distribution.
  arXiv  Detail & Related papers  (2025-07-03T14:36:15Z)
• Private Training & Data Generation by Clustering Embeddings [74.00687214400021]
  Differential privacy (DP) provides a robust framework for protecting individual data.<n>We introduce a novel principled method for DP synthetic image embedding generation.<n> Empirically, a simple two-layer neural network trained on synthetically generated embeddings achieves state-of-the-art (SOTA) classification accuracy.
  arXiv  Detail & Related papers  (2025-06-20T00:17:14Z)
• Little Giants: Synthesizing High-Quality Embedding Data at Scale [71.352883755806]
  We introduce SPEED, a framework that aligns open-source small models to efficiently generate large-scale embedding data. SPEED uses only less than 1/10 of the GPT API calls, outperforming the state-of-the-art embedding model E5_mistral when both are trained solely on their synthetic data.
  arXiv  Detail & Related papers  (2024-10-24T10:47:30Z)
• Private prediction for large-scale synthetic text generation [28.488459921169905]
  We present an approach for generating differentially private synthetic text using large language models (LLMs) In the private prediction framework, we only require the output synthetic data to satisfy differential privacy guarantees.
  arXiv  Detail & Related papers  (2024-07-16T18:28:40Z)
• Differentially-Private Data Synthetisation for Efficient Re-Identification Risk Control [3.8811062755861956]
  $epsilon$-PrivateSMOTE is a technique for safeguarding against re-identification and linkage attacks. Our proposal combines synthetic data generation via noise-induced adversarial with differential privacy principles to obfuscate high-risk cases.
  arXiv  Detail & Related papers  (2022-12-01T13:20:37Z)
• Private Set Generation with Discriminative Information [63.851085173614]
  Differentially private data generation is a promising solution to the data privacy challenge. Existing private generative models are struggling with the utility of synthetic samples. We introduce a simple yet effective method that greatly improves the sample utility of state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-07T10:02:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.