When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training

• URL: http://arxiv.org/abs/2607.13541v1
• Date: Wed, 15 Jul 2026 07:44:14 GMT
• Title: When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Training
• Abstract summary: This work reveals, for the first time, that RSMT can substantially amplify privacy leakage of real training samples.<n>We propose RSMixLeak to systematically assess this risk through membership inference attacks.
• Score: 26.069486374103338
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To overcome data scarcity and privacy constraints in data collection, it has become standard practice across academia and industry to augment real training data with text-to-image (T2I)-generated synthetic data, a paradigm we term Real-Synthetic Mix-Training (RSMT). While substituting synthetic data for sensitive real samples is widely regarded as a means to mitigate privacy exposure of the substituted data, the risk to the remaining real samples that actively participate in training has remained largely unexamined. This work reveals, for the first time, that RSMT can substantially amplify privacy leakage of these real training samples. We establish a theoretical framework, RSMT Memorization Amplification, proving that incorporating synthetic data displaces real samples toward peripheral regions of the mixed feature space, in turn forcing the model to memorize them more aggressively. Guided by this foundation, we propose RSMixLeak to systematically assess this risk through membership inference attacks (MIAs). RSMixLeak comprises two variants depending on the adversary's capability. The non-adversarial variant audits a benign RSMT pipeline with an honest T2I provider, establishing a lower bound on the leakage induced by the intrinsic gap between real and T2I-generated data. The adversarial variant considers an adversary who controls the T2I model or contributes crafted data to the T2I provider, and deliberately enlarges this distributional gap on a target class via either high-level semantic attribute binding or imperceptible pixel-level coating, further amplifying leakage on real training data while improving downstream model utility. Motivated by these findings, we further propose a lightweight leakage propensity indicator computable from real data alone that reliably identifies high-risk datasets unsuitable for entering RSMT, as a self-assessable mitigation.

Related papers

• Learning with Synthetic Data via SGD in High-Dimensional Linear Regression [29.71683151194804]
  We study how synthetic data affects the generalization of one-pass SGD in high-dimensional linear regression with model shift.<n>Mixed training induces strong model collapse, while two-stage training avoids the floor by using synthetic data only in the first stage.
  arXiv  Detail & Related papers  (2026-09-09T00:53:32Z)
• Should I Use This Synthetic Dataset for Training? How to Test with Minimal Real Data [73.19814469893088]
  Digital twins and learned world models are increasingly used to generate synthetic data that augment the scarce real datasets available for training artificial intelligence (AI) models in engineering systems.<n>This paper addresses whether training on the augmented dataset improves the true, population-level performance, while consuming as few real test data points as possible.<n>Experiments on a synthetic-data classification task, a DT-aided wireless packet-scheduling task, and a radio-map prediction task show that aeSFT identifies useful synthetic data using substantially fewer real test samples than mean-based sequential testing.
  arXiv  Detail & Related papers  (2026-08-28T07:05:38Z)
• Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data [56.86147283213652]
  We present a customizable empirical auditing framework designed to detect and explain data disclosures.<n>Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures"-where the system incidentally generates a user's data.
  arXiv  Detail & Related papers  (2026-06-15T16:54:02Z)
• Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning [8.35064965629059]
  Multimodal Federated Learning (MMFL) enables privacy-preserving collaborative training.<n>Real-world clinical applications often suffer from within-modality missingness caused by sensor intermittency or irregular sampling.<n>We propose CondI, a framework explicitly addressing this missingness using conditional diffusion models.
  arXiv  Detail & Related papers  (2026-04-25T02:35:08Z)
• PuckTrick: A Library for Making Synthetic Data More Realistic [46.198289193451146]
  We introduce Pucktrick, a Python library designed to systematically contaminate synthetic datasets by introducing controlled errors.<n>We evaluate the impact of systematic data contamination on model performance.<n>Our findings demonstrate that ML models trained on contaminated synthetic data outperform those trained on purely synthetic, error-free data.
  arXiv  Detail & Related papers  (2025-06-23T10:51:45Z)
• Unlocking Post-hoc Dataset Inference with Synthetic Data [11.886166976507711]
  Training datasets are often scraped from the internet without respecting data owners' intellectual property rights.<n>Inference (DI) offers a potential remedy by identifying whether a suspect dataset was used in training.<n>Existing DI methods require a private set-known to be absent from training-that closely matches the compromised dataset's distribution.<n>In this work, we address this challenge by synthetically generating the required held-out set.
  arXiv  Detail & Related papers  (2025-06-18T08:46:59Z)
• LSM-2: Learning from Incomplete Wearable Sensor Data [65.58595667477505]
  This paper introduces the second generation of Large Sensor Model (LSM-2) with Adaptive and Inherited Masking (AIM)<n>AIM learns robust representations directly from incomplete data without requiring explicit imputation.<n>Our LSM-2 with AIM achieves the best performance across a diverse range of tasks, including classification, regression and generative modeling.
  arXiv  Detail & Related papers  (2025-06-05T17:57:11Z)
• VAE-based Feature Disentanglement for Data Augmentation and Compression in Generalized GNSS Interference Classification [42.14439854721613]
  We propose variational autoencoders (VAEs) for disentanglement to extract essential latent features that enable accurate classification of interferences.<n>Our proposed VAE achieves a data compression rate ranging from 512 to 8,192 and achieves an accuracy up to 99.92%.
  arXiv  Detail & Related papers  (2025-04-14T13:38:00Z)
• D2SP: Dynamic Dual-Stage Purification Framework for Dual Noise Mitigation in Vision-based Affective Recognition [32.74206402632733]
  Noise arises from low-quality captures that defy logical labeling, and instances that suffer from mislabeling due to annotation bias. We have crafted a two-stage framework aiming at textbfSeeking textbfCertain data textbfIn extensive textbfUncertain data (SCIU) This initiative aims to purge the DFER datasets of these uncertainties, thereby ensuring that only clean, verified data is employed in training processes.
  arXiv  Detail & Related papers  (2024-06-24T09:25:02Z)
• Towards Assessing the Synthetic-to-Measured Adversarial Vulnerability of SAR ATR [16.144102386839574]
  This paper studies the synthetic-to-measured (S2M) transfer setting, where an attacker generates adversarial perturbation based solely on synthetic data and transfers it against victim models trained with measured data. We also propose the transferability estimation attack (TEA) to uncover the adversarial risks in this more challenging and practical scenario.
  arXiv  Detail & Related papers  (2024-01-30T14:16:24Z)
• From Fake to Real: Pretraining on Balanced Synthetic Images to Prevent Spurious Correlations in Image Recognition [64.59093444558549]
  We propose a simple, easy-to-implement, two-step training pipeline that we call From Fake to Real. By training on real and synthetic data separately, FFR does not expose the model to the statistical differences between real and synthetic data. Our experiments show that FFR improves worst group accuracy over the state-of-the-art by up to 20% over three datasets.
  arXiv  Detail & Related papers  (2023-08-08T19:52:28Z)
• Uncertain Facial Expression Recognition via Multi-task Assisted Correction [43.02119884581332]
  We propose a novel method of multi-task assisted correction in addressing uncertain facial expression recognition called MTAC. Specifically, a confidence estimation block and a weighted regularization module are applied to highlight solid samples and suppress uncertain samples in every batch. Experiments on RAF-DB, AffectNet, and AffWild2 datasets demonstrate that the MTAC obtains substantial improvements over baselines when facing synthetic and real uncertainties.
  arXiv  Detail & Related papers  (2022-12-14T10:28:08Z)
• Concealing Sensitive Samples against Gradient Leakage in Federated Learning [41.43099791763444]
  Federated Learning (FL) is a distributed learning paradigm that enhances users privacy by eliminating the need for clients to share raw, private data with the server. Recent studies expose the vulnerability of FL to model inversion attacks, where adversaries reconstruct users private data via eavesdropping on the shared gradient information. We present a simple, yet effective defense strategy that obfuscates the gradients of the sensitive data with concealed samples.
  arXiv  Detail & Related papers  (2022-09-13T04:19:35Z)
• Disentangled Representation Learning for RF Fingerprint Extraction under Unknown Channel Statistics [77.13542705329328]
  We propose a framework of disentangled representation learning(DRL) that first learns to factor the input signals into a device-relevant component and a device-irrelevant component via adversarial learning. The implicit data augmentation in the proposed framework imposes a regularization on the RFF extractor to avoid the possible overfitting of device-irrelevant channel statistics. Experiments validate that the proposed approach, referred to as DR-RFF, outperforms conventional methods in terms of generalizability to unknown complicated propagation environments.
  arXiv  Detail & Related papers  (2022-08-04T15:46:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.