On the Impact of Anonymization on the Performance of Large Language Models

• URL: http://arxiv.org/abs/2609.11335v2
• Date: Tue, 15 Sep 2026 11:09:08 GMT
• Title: On the Impact of Anonymization on the Performance of Large Language Models
• Abstract summary: This paper presents a systematic empirical study of the trade-off between privacy and performance.<n>We evaluate five prominent language models across eleven diverse benchmarks.<n>We find that while anonymization generally degrades performance, the effect is highly nuanced.
• Score: 3.812539490062037
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As large language models are increasingly deployed in sensitive domains, anonymizing input data to protect personally identifiable information has become a critical practice. However, the impact of this anonymization on model utility is not well understood. This paper presents a systematic empirical study of the trade-off between privacy and performance. We evaluate five prominent language models across eleven diverse benchmarks, comparing their performance on original versus pseudonymized inputs. Our results reveal that while anonymization generally degrades performance, the effect is highly nuanced. We find that more capable models, such as Qwen2.5-72B and GPT-4o mini, suffer the largest performance drops, suggesting a stronger reliance on specific entity information. The impact is also task-dependent: performance on TruthfulQA improves with anonymization, while retrieval-focused tasks like RGB experience a catastrophic decline. Further experiments show that reversible anonymization techniques that preserve entity uniqueness significantly outperform irreversible ones like redaction, and that explicitly prompting models about anonymization offers no discernible benefit. We conclude that anonymization is not a one-size-fits-all solution and must be co-designed with the model and task in mind to balance privacy and utility effectively. Our findings provide a crucial baseline for developing more robust, privacy-aware AI systems.

Related papers

• Evaluating the Impact of Data Anonymization on Image Retrieval [2.8165396388850112]
  We study the impact of anonymization on Content-Based Image Retrieval (CBIR) using two public datasets and the internal DOKIQ dataset.<n>Our results reveal a pronounced retrieval bias in favor of models trained on original data, which produce the most similar retrievals after anonymization.
  arXiv  Detail & Related papers  (2026-02-23T09:39:06Z)
• AgentStealth: Reinforcing Large Language Model for Anonymizing User-generated Text [8.758843436588297]
  AgentStealth is a self-reinforcing language model for text anonymization.<n>We show that our method outperforms baselines in both anonymization effectiveness and utility.<n>Our lightweight design supports direct deployment on edge devices, avoiding cloud reliance and communication-based privacy risks.
  arXiv  Detail & Related papers  (2025-06-26T02:48:16Z)
• Self-Refining Language Model Anonymizers via Adversarial Distillation [48.280759014096354]
  We introduce SElf-refining Anonymization with Language model (SEAL)<n>SEAL is a novel distillation framework for training small language models (SLMs) to perform effective anonymization without relying on external models at inference time.<n>Experiments on SynthPAI, a dataset of synthetic personal profiles and text comments, demonstrate that SLMs trained with SEAL achieve substantial improvements in anonymization capabilities.
  arXiv  Detail & Related papers  (2025-06-02T08:21:27Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Anonymizing text that contains sensitive information is crucial for a wide range of applications.<n>Existing techniques face the emerging challenges of the re-identification ability of large language models.<n>We propose a framework composed of three key components: a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Large Language Models are Advanced Anonymizers [2.9373912230684565]
  Recent privacy research on large language models (LLMs) has shown that they achieve near-human-level performance at inferring personal data from online texts.<n>Existing text anonymization methods are currently lacking behind regulatory requirements and adversarial threats.<n>We present a new setting for evaluating anonymization in the face of adversarial LLM inferences.
  arXiv  Detail & Related papers  (2024-02-21T14:44:00Z)
• Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification [55.741525129613535]
  "Disentangle Before Anonymize" is a novel two-stage Framework(DBAF)<n>This framework includes a Contrastive Identity Disentanglement (CID) module and a Key-authorized Reversible Identity Anonymization (KRIA) module.<n>Extensive experiments demonstrate that our method outperforms state-of-the-art de-identification approaches.
  arXiv  Detail & Related papers  (2023-11-15T08:59:02Z)
• Locally Differentially Private Document Generation Using Zero Shot Prompting [61.20953109732442]
  We propose a locally differentially private mechanism called DP-Prompt to counter author de-anonymization attacks. When DP-Prompt is used with a powerful language model like ChatGPT (gpt-3.5), we observe a notable reduction in the success rate of de-anonymization attacks.
  arXiv  Detail & Related papers  (2023-10-24T18:25:13Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Does Image Anonymization Impact Computer Vision Training? [0.0]
  We investigate the impact of image anonymization for training computer vision models on key computer vision tasks. Traditional image anonymization substantially impacts final model performance, particularly when anonymizing the full body. We find that realistic anonymization can mitigate this decrease in performance, where our experiments reflect a minimal performance drop for face anonymization.
  arXiv  Detail & Related papers  (2023-06-08T12:02:03Z)
• GANonymization: A GAN-based Face Anonymization Framework for Preserving Emotional Expressions [43.017036538109274]
  GANonymization is a novel face anonymization framework with facial expression-preserving abilities. Our approach is based on a high-level representation of a face, which is synthesized into an anonymized version based on a generative adversarial network (GAN)
  arXiv  Detail & Related papers  (2023-05-03T14:22:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.