Reconstruction of Personally Identifiable Information from Supervised Finetuned Models

• URL: http://arxiv.org/abs/2605.12264v1
• Date: Tue, 12 May 2026 15:29:33 GMT
• Title: Reconstruction of Personally Identifiable Information from Supervised Finetuned Models
• Abstract summary: Supervised Finetuning (SFT) has become one of the primary methods for adapting a large language model.<n>This paper studies the problem of PII reconstruction from SFT models for the first time.
• Score: 7.127816156842457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Supervised Finetuning (SFT) has become one of the primary methods for adapting a large language model (LLM) with extensive pre-trained knowledge to domain-specific, instruction-following tasks. SFT datasets, composed of instruction-response pairs, often include user-provided information that may contain sensitive data such as personally identifiable information (PII), raising privacy concerns. This paper studies the problem of PII reconstruction from SFT models for the first time. We construct multi-turn, user-centric Q&A datasets in sensitive domains, specifically medical and legal settings, that incorporate PII to enable realistic evaluation of leakage. Using these datasets, we evaluate the extent to which an adversary, with varying levels of knowledge about the fine-tuning dataset, can infer sensitive information about individuals whose data was used during SFT. In the reconstruction setting, we propose COVA, a novel decoding algorithm to reconstruct PII under prefix-based attacks, consistently outperforming existing extraction methods. Our results show that even partial attacker knowledge can significantly improve reconstruction success, while leakage varies substantially across PII types.

Related papers

• AI-Based Culvert-Sewer Inspection [0.0]
  Culverts and sewer pipes are critical components of drainage systems, and their failure can lead to serious risks to public safety and the environment.<n>This thesis proposes three methods to significantly enhance defect segmentation and handle data scarcity.<n>ForTRESS is a novel architecture that combines depthwise separable convolutions, adaptive Kolmogorov-Arnold Networks (KAN), and multi-scale attention mechanisms.
  arXiv  Detail & Related papers  (2026-01-21T16:33:33Z)
• Understanding Privacy Risks in Code Models Through Training Dynamics: A Causal Approach [58.05800140178267]
  Large language models for code (LLM4Code) have greatly improved developer productivity but also raise privacy concerns.<n>We investigate whether distinct PII types vary in their likelihood of being learned and leaked by LLM4Code.<n>Results show that leakage risks differ substantially across PII types and correlate with their training dynamics.<n>This work provides the first causal evidence that leakage risks are type-dependent and offers guidance for developing type-aware and learnability-aware defenses.
  arXiv  Detail & Related papers  (2025-12-08T18:47:40Z)
• Knowledge Abstraction for Knowledge-based Semantic Communication: A Generative Causality Invariant Approach [4.09282702124671]
  We propose a generative adversarial network that leverages causality-invariant learning to extract causal and non-causal representations from the data.<n> causality-invariant knowledge ensures consistency across different devices despite the diverse training data.<n>Our knowledge-based data reconstruction highlights the robustness of our decoder, which surpasses other state-of-the-art data reconstruction and semantic compression methods in terms of Peak Signal-to-Noise Ratio (PSNR)
  arXiv  Detail & Related papers  (2025-07-23T06:56:07Z)
• Resolving Knowledge Conflicts in Domain-specific Data Selection: A Case Study on Medical Instruction-tuning [83.99974309930072]
  Domain-specific instruction-tuning has become the defacto standard for improving the performance of large language models.<n>We propose a Knowledge-aware Data Selection framework to select the domain-specific instruction-tuning data that meets LLMs' actual needs.<n>By filtering the data with large knowledge conflicts and sampling the high-quality and diverse data, KDS can effectively stimulate the LLMs' abilities and achieve better domain-specific performance.
  arXiv  Detail & Related papers  (2025-05-28T04:18:24Z)
• AdvKT: An Adversarial Multi-Step Training Framework for Knowledge Tracing [64.79967583649407]
  Knowledge Tracing (KT) monitors students' knowledge states and simulates their responses to question sequences.<n>Existing KT models typically follow a single-step training paradigm, which leads to significant error accumulation.<n>We propose a novel Adversarial Multi-Step Training Framework for Knowledge Tracing (AdvKT) which focuses on the multi-step KT task.
  arXiv  Detail & Related papers  (2025-04-07T03:31:57Z)
• A Dataset for Semantic Segmentation in the Presence of Unknowns [49.795683850385956]
  Existing datasets allow evaluation of only knowns or unknowns - but not both.<n>We propose a novel anomaly segmentation dataset, ISSU, that features a diverse set of anomaly inputs from cluttered real-world environments.<n>The dataset is twice larger than existing anomaly segmentation datasets.
  arXiv  Detail & Related papers  (2025-03-28T10:31:01Z)
• Capturing the Temporal Dependence of Training Data Influence [100.91355498124527]
  We formalize the concept of trajectory-specific leave-one-out influence, which quantifies the impact of removing a data point during training.<n>We propose data value embedding, a novel technique enabling efficient approximation of trajectory-specific LOO.<n>As data value embedding captures training data ordering, it offers valuable insights into model training dynamics.
  arXiv  Detail & Related papers  (2024-12-12T18:28:55Z)
• Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data [18.984529269623135]
  This study investigates whether fine-tuning with generated data truly enhances privacy or introduces additional privacy risks.<n>We use the Pythia Model Suite and Open Pre-trained Transformer to measure privacy risks.
  arXiv  Detail & Related papers  (2024-09-12T10:14:12Z)
• QBI: Quantile-Based Bias Initialization for Efficient Private Data Reconstruction in Federated Learning [0.5497663232622965]
  Federated learning enables the training of machine learning models on distributed data without compromising user privacy. Recent research has shown that the central entity can perfectly reconstruct private data from shared model updates.
  arXiv  Detail & Related papers  (2024-06-26T20:19:32Z)
• Visual Privacy Auditing with Diffusion Models [47.0700328585184]
  We introduce a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors.<n>We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and DMs can serve as auditing tools for visualizing privacy leakage.
  arXiv  Detail & Related papers  (2024-03-12T12:18:55Z)
• Layer-wise Characterization of Latent Information Leakage in Federated Learning [9.397152006395174]
  Training deep neural networks via federated learning allows clients to share, instead of the original data, only the model trained on their data. Prior work has demonstrated that in practice a client's private information, unrelated to the main learning task, can be discovered from the model's gradients. There is still no formal approach for quantifying the leakage of private information via the shared updated model or gradients.
  arXiv  Detail & Related papers  (2020-10-17T10:49:14Z)
• Representation Learning for Sequence Data with Deep Autoencoding Predictive Components [96.42805872177067]
  We propose a self-supervised representation learning method for sequence data, based on the intuition that useful representations of sequence data should exhibit a simple structure in the latent space. We encourage this latent structure by maximizing an estimate of predictive information of latent feature sequences, which is the mutual information between past and future windows at each time step. We demonstrate that our method recovers the latent space of noisy dynamical systems, extracts predictive features for forecasting tasks, and improves automatic speech recognition when used to pretrain the encoder on large amounts of unlabeled data.
  arXiv  Detail & Related papers  (2020-10-07T03:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.