The Conundrum of Trustworthy Research on Attacking Personally Identifiable Information Removal Techniques

• URL: http://arxiv.org/abs/2603.08207v1
• Date: Mon, 09 Mar 2026 10:38:43 GMT
• Title: The Conundrum of Trustworthy Research on Attacking Personally Identifiable Information Removal Techniques
• Abstract summary: We show that documents sanitized by PII removal techniques are vulnerable to reconstruction attacks.<n>We critically analyze the evaluation of existing attacks and find that data leakage and data contamination are not properly mitigated.<n>We conclude that only truly private data can allow us to objectively evaluate vulnerabilities in PII removal techniques.
• Score: 12.725838159837288
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Removing personally identifiable information (PII) from texts is necessary to comply with various data protection regulations and to enable data sharing without compromising privacy. However, recent works show that documents sanitized by PII removal techniques are vulnerable to reconstruction attacks. Yet, we suspect that the reported success of these attacks is largely overestimated. We critically analyze the evaluation of existing attacks and find that data leakage and data contamination are not properly mitigated, leaving the question whether or not PII removal techniques truly protect privacy in real-world scenarios unaddressed. We investigate possible data sources and attack setups that avoid data leakage and conclude that only truly private data can allow us to objectively evaluate vulnerabilities in PII removal techniques. However, access to private data is heavily restricted - and for good reasons - which also means that the public research community cannot address this problem in a transparent, reproducible, and trustworthy manner.

Related papers

• DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective [70.77570343385928]
  We introduce a novel taxonomy, classifying existing methods based on their reliance on internal features (IF) (inherent to the data) versus external features (EF) (artificially introduced for auditing)<n>We formulate two primary attack types: evasion attacks, designed to conceal the use of a dataset, and forgery attacks, intending to falsely implicate an unused dataset.<n>Building on the understanding of existing methods and attack objectives, we further propose systematic attack strategies: decoupling, removal, and detection for evasion; adversarial example-based methods for forgery.<n>Our benchmark, DATABench, comprises 17 evasion attacks, 5 forgery attacks, and 9
  arXiv  Detail & Related papers  (2025-07-08T03:07:15Z)
• A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage [77.83757117924995]
  We propose a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release.<n>Our approach shows that seemingly innocuous auxiliary information can be used to infer sensitive attributes like age or substance use history from sanitized data.
  arXiv  Detail & Related papers  (2025-04-28T01:16:27Z)
• Investigating Vulnerabilities of GPS Trip Data to Trajectory-User Linking Attacks [49.1574468325115]
  We propose a novel attack to reconstruct user identifiers in GPS trip datasets consisting of single trips.<n>We show that the risk of re-identification is significant even when personal identifiers have been removed.<n>Further investigations indicate that users who frequently visit locations that are only visited by a small number of others tend to be more vulnerable to re-identification.
  arXiv  Detail & Related papers  (2025-02-12T08:54:49Z)
• Enforcing Demographic Coherence: A Harms Aware Framework for Reasoning about Private Data Release [14.939460540040459]
  We introduce demographic coherence, a condition inspired by privacy attacks that we argue is necessary for data privacy.<n>Our framework focuses on confidence rated predictors, which can in turn be distilled from almost any data-informed process.<n>We prove that every differentially private data release is also demographically coherent, and that there are demographically coherent algorithms which are not differentially private.
  arXiv  Detail & Related papers  (2025-02-04T20:42:30Z)
• Demystifying Trajectory Recovery From Ash: An Open-Source Evaluation and Enhancement [5.409124675229009]
  This study reimplements the trajectory recovery attack from scratch and evaluates it on two open-source datasets. Results confirm that privacy leakage still exists despite common anonymisation and aggregation methods. We propose a stronger attack by designing a series of enhancements to the baseline attack.
  arXiv  Detail & Related papers  (2024-09-23T01:06:41Z)
• Visual Privacy Auditing with Diffusion Models [47.0700328585184]
  We introduce a reconstruction attack based on diffusion models (DMs) that only assumes adversary access to real-world image priors.<n>We find that (1) real-world data priors significantly influence reconstruction success, (2) current reconstruction bounds do not model the risk posed by data priors well, and DMs can serve as auditing tools for visualizing privacy leakage.
  arXiv  Detail & Related papers  (2024-03-12T12:18:55Z)
• A Summary of Privacy-Preserving Data Publishing in the Local Setting [0.6749750044497732]
  Statistical Disclosure Control aims to minimize the risk of exposing confidential information by de-identifying it. We outline the current privacy-preserving techniques employed in microdata de-identification, delve into privacy measures tailored for various disclosure scenarios, and assess metrics for information loss and predictive performance.
  arXiv  Detail & Related papers  (2023-12-19T04:23:23Z)
• Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks [70.39633252935445]
  Data contamination has become prevalent and challenging with the rise of models pretrained on large automatically-crawled corpora. For closed models, the training data becomes a trade secret, and even for open models, it is not trivial to detect contamination. We propose three strategies that can make a difference: (1) Test data made public should be encrypted with a public key and licensed to disallow derivative distribution; (2) demand training exclusion controls from closed API holders, and protect your test data by refusing to evaluate without them; and (3) avoid data which appears with its solution on the internet, and release the web-page context of internet-derived
  arXiv  Detail & Related papers  (2023-05-17T12:23:38Z)
• No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy" [75.98836424725437]
  New methods designed to preserve data privacy require careful scrutiny. Failure to preserve privacy is hard to detect, and yet can lead to catastrophic results when a system implementing a privacy-preserving'' method is attacked.
  arXiv  Detail & Related papers  (2022-09-29T17:50:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.