Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization

• URL: http://arxiv.org/abs/2506.05979v1
• Date: Fri, 06 Jun 2025 10:59:35 GMT
• Title: Tau-Eval: A Unified Evaluation Framework for Useful and Private Text Anonymization
• Authors: Gabriel Loiseau, Damien Sileo, Damien Riquet, Maxime Meyer, Marc Tommasi,
• Abstract summary: We present Tau-Eval, an open-source framework for benchmarking text anonymization methods.<n>A Python library, code, documentation and tutorials are publicly available.
• Score: 5.239989658197324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text anonymization is the process of removing or obfuscating information from textual data to protect the privacy of individuals. This process inherently involves a complex trade-off between privacy protection and information preservation, where stringent anonymization methods can significantly impact the text's utility for downstream applications. Evaluating the effectiveness of text anonymization proves challenging from both privacy and utility perspectives, as there is no universal benchmark that can comprehensively assess anonymization techniques across diverse, and sometimes contradictory contexts. We present Tau-Eval, an open-source framework for benchmarking text anonymization methods through the lens of privacy and utility task sensitivity. A Python library, code, documentation and tutorials are publicly available.

Related papers

• Self-Refining Language Model Anonymizers via Adversarial Distillation [49.17383264812234]
  Large language models (LLMs) are increasingly used in sensitive domains, where their ability to infer personal data poses emerging privacy risks.<n>We introduce SElf-refining Anonymization with Language model (SEAL), a novel distillation framework for training small language models (SLMs) to perform effective anonymization.
  arXiv  Detail & Related papers  (2025-06-02T08:21:27Z)
• A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage [77.83757117924995]
  We propose a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release.<n>Our approach shows that seemingly innocuous auxiliary information can be used to infer sensitive attributes like age or substance use history from sanitized data.
  arXiv  Detail & Related papers  (2025-04-28T01:16:27Z)
• Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy [64.32494202656801]
  Privacy-preserving computer vision is an important emerging problem in machine learning and artificial intelligence.<n>We present anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context.<n>We also proposeMaskDP to protect non-anonymized but privacy sensitive background information.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• Robust Utility-Preserving Text Anonymization Based on Large Language Models [80.5266278002083]
  Text anonymization is crucial for sharing sensitive data while maintaining privacy. Existing techniques face the emerging challenges of re-identification attack ability of Large Language Models. This paper proposes a framework composed of three LLM-based components -- a privacy evaluator, a utility evaluator, and an optimization component.
  arXiv  Detail & Related papers  (2024-07-16T14:28:56Z)
• IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization [8.483679748399037]
  We propose IncogniText, a technique that anonymizes the text to mislead a potential adversary into predicting a wrong private attribute value.<n>Our empirical evaluation shows a reduction of private attribute leakage by more than 90% across 8 different private attributes.
  arXiv  Detail & Related papers  (2024-07-03T09:49:03Z)
• NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human [56.46355425175232]
  We suggest sanitizing sensitive text using two common strategies used by humans.<n>We curate the first corpus, coined NAP2, through both crowdsourcing and the use of large language models.<n>Compared to the prior works on anonymization, the human-inspired approaches result in more natural rewrites.
  arXiv  Detail & Related papers  (2024-06-06T05:07:44Z)
• RedactBuster: Entity Type Recognition from Redacted Documents [13.172863061928899]
  We propose RedactBuster, the first deanonymization model using sentence context to perform Named Entity Recognition on reacted text. We test RedactBuster against the most effective redaction technique and evaluate it using the publicly available Text Anonymization Benchmark (TAB) Our results show accuracy values up to 0.985 regardless of the document nature or entity type.
  arXiv  Detail & Related papers  (2024-04-19T16:42:44Z)
• SPAct: Self-supervised Privacy Preservation for Action Recognition [73.79886509500409]
  Existing approaches for mitigating privacy leakage in action recognition require privacy labels along with the action labels from the video dataset. Recent developments of self-supervised learning (SSL) have unleashed the untapped potential of the unlabeled data. We present a novel training framework which removes privacy information from input video in a self-supervised manner without requiring privacy labels.
  arXiv  Detail & Related papers  (2022-03-29T02:56:40Z)
• The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization [2.9849405664643585]
  We present a novel benchmark and associated evaluation metrics for assessing the performance of text anonymization methods. Text anonymization, defined as the task of editing a text document to prevent the disclosure of personal information, currently suffers from a shortage of privacy-oriented annotated text resources. This paper presents TAB (Text Anonymization Benchmark), a new, open-source annotated corpus developed to address this shortage.
  arXiv  Detail & Related papers  (2022-01-25T14:34:42Z)
• Protecting Anonymous Speech: A Generative Adversarial Network Methodology for Removing Stylistic Indicators in Text [2.9005223064604078]
  We develop a new approach to authorship anonymization by constructing a generative adversarial network. Our fully automatic method achieves comparable results to other methods in terms of content preservation and fluency. Our approach is able to generalize well to an open-set context and anonymize sentences from authors it has not encountered before.
  arXiv  Detail & Related papers  (2021-10-18T17:45:56Z)
• No Intruder, no Validity: Evaluation Criteria for Privacy-Preserving Text Anonymization [0.48733623015338234]
  We argue that researchers and practitioners developing automated text anonymization systems should carefully assess whether their evaluation methods truly reflect the system's ability to protect individuals from being re-identified. We propose TILD, a set of evaluation criteria that comprises an anonymization method's technical performance, the information loss resulting from its anonymization, and the human ability to de-anonymize redacted documents.
  arXiv  Detail & Related papers  (2021-03-16T18:18:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.