Interpretable Privacy Preservation of Text Representations Using Vector Steganography

• URL: http://arxiv.org/abs/2112.02557v2
• Date: Tue, 7 Dec 2021 10:22:52 GMT
• Title: Interpretable Privacy Preservation of Text Representations Using Vector Steganography
• Authors: Geetanjali Bihani
• Abstract summary: Contextual word representations generated by language models (LMs) learn spurious associations present in the training corpora. adversaries can exploit these associations to reverse-engineer the private attributes of entities mentioned within the corpora. I aim to study and develop methods to incorporate steganographic modifications within the vector geometry to obfuscate underlying spurious associations.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contextual word representations generated by language models (LMs) learn spurious associations present in the training corpora. Recent findings reveal that adversaries can exploit these associations to reverse-engineer the private attributes of entities mentioned within the corpora. These findings have led to efforts towards minimizing the privacy risks of language models. However, existing approaches lack interpretability, compromise on data utility and fail to provide privacy guarantees. Thus, the goal of my doctoral research is to develop interpretable approaches towards privacy preservation of text representations that retain data utility while guaranteeing privacy. To this end, I aim to study and develop methods to incorporate steganographic modifications within the vector geometry to obfuscate underlying spurious associations and preserve the distributional semantic properties learnt during training.

Related papers

• Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding [118.75567341513897]
  Existing methods typically analyze target text in isolation or solely with non-member contexts. We propose Con-ReCall, a novel approach that leverages the asymmetric distributional shifts induced by member and non-member contexts.
  arXiv  Detail & Related papers  (2024-09-05T09:10:38Z)
• NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human [55.20137833039499]
  We suggest sanitizing sensitive text using two common strategies used by humans. We curate the first corpus, coined NAP2, through both crowdsourcing and the use of large language models.
  arXiv  Detail & Related papers  (2024-06-06T05:07:44Z)
• Privacy-Preserving Language Model Inference with Instance Obfuscation [33.86459812694288]
  Language Models as a Service (LM) offers convenient access for developers and researchers to perform inference using pre-trained language models. The input data and the inference results containing private information are exposed as plaintext during the service call, leading to privacy issues. We propose Instance-Obfuscated Inference (IOI) method, which focuses on addressing the decision privacy issue of natural language understanding tasks.
  arXiv  Detail & Related papers  (2024-02-13T05:36:54Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• Disentangling the Linguistic Competence of Privacy-Preserving BERT [0.0]
  Differential Privacy (DP) has been tailored to address the unique challenges of text-to-text privatization. We employ a series of interpretation techniques on the internal representations extracted from BERT trained on perturbed pre-text. Using probing tasks to unpack this dissimilarity, we find evidence that text-to-text privatization affects the linguistic competence across several formalisms.
  arXiv  Detail & Related papers  (2023-10-17T16:00:26Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [56.85319224208865]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account. We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed. Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• What Does it Mean for a Language Model to Preserve Privacy? [12.955456268790005]
  Natural language reflects our private lives and identities, making its privacy concerns as broad as those of real life. We argue that existing protection methods cannot guarantee a generic and meaningful notion of privacy for language models. We conclude that language models should be trained on text data which was explicitly produced for public use.
  arXiv  Detail & Related papers  (2022-02-11T09:18:27Z)
• Semantics-Preserved Distortion for Personal Privacy Protection in Information Management [65.08939490413037]
  This paper suggests a linguistically-grounded approach to distort texts while maintaining semantic integrity. We present two distinct frameworks for semantic-preserving distortion: a generative approach and a substitutive approach. We also explore privacy protection in a specific medical information management scenario, showing our method effectively limits sensitive data memorization.
  arXiv  Detail & Related papers  (2022-01-04T04:01:05Z)
• CAPE: Context-Aware Private Embeddings for Private Language Learning [0.5156484100374058]
  Context-Aware Private Embeddings (CAPE) is a novel approach which preserves privacy during training of embeddings. CAPE applies calibrated noise through differential privacy, preserving the encoded semantic links while obscuring sensitive information. Experimental results demonstrate that the proposed approach reduces private information leakage better than either single intervention.
  arXiv  Detail & Related papers  (2021-08-27T14:50:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.