Differential Privacy for Text Analytics via Natural Text Sanitization

• URL: http://arxiv.org/abs/2106.01221v1
• Date: Wed, 2 Jun 2021 15:15:10 GMT
• Title: Differential Privacy for Text Analytics via Natural Text Sanitization
• Authors: Xiang Yue, Minxin Du, Tianhao Wang, Yaliang Li, Huan Sun and Sherman S. M. Chow
• Abstract summary: This paper takes a direct approach to text sanitization. Our insight is to consider both sensitivity and similarity via our new local DP notion. The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacy-preserving natural language processing over the BERT language model with promising utility.
• Score: 44.95170585853761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Texts convey sophisticated knowledge. However, texts also convey sensitive information. Despite the success of general-purpose language models and domain-specific mechanisms with differential privacy (DP), existing text sanitization mechanisms still provide low utility, as cursed by the high-dimensional text representation. The companion issue of utilizing sanitized texts for downstream analytics is also under-explored. This paper takes a direct approach to text sanitization. Our insight is to consider both sensitivity and similarity via our new local DP notion. The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacy-preserving natural language processing over the BERT language model with promising utility. Surprisingly, the high utility does not boost up the success rate of inference attacks.

Related papers

• On the Vulnerability of Text Sanitization [60.162007426724564]
  We propose theoretically optimal reconstruction attacks targeting text sanitization. We derive their bounds on ASR as benchmarks for evaluating sanitization performance. One of our attacks achieves a 46.4% improvement in ASR over the state-of-the-art baseline.
  arXiv  Detail & Related papers  (2024-10-22T14:31:53Z)
• NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human [55.20137833039499]
  We suggest sanitizing sensitive text using two common strategies used by humans. We curate the first corpus, coined NAP2, through both crowdsourcing and the use of large language models.
  arXiv  Detail & Related papers  (2024-06-06T05:07:44Z)
• Efficiently Leveraging Linguistic Priors for Scene Text Spotting [63.22351047545888]
  This paper proposes a method that leverages linguistic knowledge from a large text corpus to replace the traditional one-hot encoding used in auto-regressive scene text spotting and recognition models. We generate text distributions that align well with scene text datasets, removing the need for in-domain fine-tuning. Experimental results show that our method not only improves recognition accuracy but also enables more accurate localization of words.
  arXiv  Detail & Related papers  (2024-02-27T01:57:09Z)
• LLM-Assisted Content Conditional Debiasing for Fair Text Embedding [37.92120550031469]
  This paper proposes a novel method for learning fair text embeddings. We define a novel content-conditional equal distance (CCED) fairness for text embeddings. We also introduce a content-conditional debiasing (CCD) loss to ensure that embeddings of texts with different sensitive attributes but identical content maintain the same distance from the embedding of their corresponding neutral text.
  arXiv  Detail & Related papers  (2024-02-22T01:20:51Z)
• Text Sanitization Beyond Specific Domains: Zero-Shot Redaction & Substitution with Large Language Models [0.0]
  We present a zero-shot text sanitization technique that detects and substitutes potentially sensitive information using Large Language Models. Our evaluation shows that our method excels at protecting privacy while maintaining text coherence and contextual information.
  arXiv  Detail & Related papers  (2023-11-16T18:42:37Z)
• Disentangling the Linguistic Competence of Privacy-Preserving BERT [0.0]
  Differential Privacy (DP) has been tailored to address the unique challenges of text-to-text privatization. We employ a series of interpretation techniques on the internal representations extracted from BERT trained on perturbed pre-text. Using probing tasks to unpack this dissimilarity, we find evidence that text-to-text privatization affects the linguistic competence across several formalisms.
  arXiv  Detail & Related papers  (2023-10-17T16:00:26Z)
• Are Your Explanations Reliable? Investigating the Stability of LIME in Explaining Text Classifiers by Marrying XAI and Adversarial Attack [11.437660523661338]
  LIME has emerged as one of the most commonly referenced tools in explainable AI (XAI) frameworks. In this paper, we first evaluate the inherent instability of LIME on text data to establish a baseline. We then propose a novel algorithm XAIFooler to perturb text inputs and manipulate explanations that casts investigation on the stability of LIME as a text perturbation optimization problem.
  arXiv  Detail & Related papers  (2023-05-21T05:06:46Z)
• Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding [80.3811072650087]
  We study natural language watermarking as a defense to help better mark and trace the provenance of text. We introduce the Adversarial Watermarking Transformer (AWT) with a jointly trained encoder-decoder and adversarial training. AWT is the first end-to-end model to hide data in text by automatically learning -- without ground truth -- word substitutions along with their locations.
  arXiv  Detail & Related papers  (2020-09-07T11:01:24Z)
• Improving Disentangled Text Representation Learning with Information-Theoretic Guidance [99.68851329919858]
  discrete nature of natural language makes disentangling of textual representations more challenging. Inspired by information theory, we propose a novel method that effectively manifests disentangled representations of text. Experiments on both conditional text generation and text-style transfer demonstrate the high quality of our disentangled representation.
  arXiv  Detail & Related papers  (2020-06-01T03:36:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.