How Private are Language Models in Abstractive Summarization?

• URL: http://arxiv.org/abs/2412.12040v1
• Date: Mon, 16 Dec 2024 18:08:22 GMT
• Title: How Private are Language Models in Abstractive Summarization?
• Authors: Anthony Hughes, Nikolaos Aletras, Ning Ma,
• Abstract summary: Language models (LMs) have shown outstanding performance in text summarization including sensitive domains such as medicine and law.<n>However, to what extent LMs can provide privacy-preserving summaries given a non-private source document remains under-explored.
• Score: 36.801842863853715
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language models (LMs) have shown outstanding performance in text summarization including sensitive domains such as medicine and law. In these settings, it is important that personally identifying information (PII) included in the source document should not leak in the summary. Prior efforts have mostly focused on studying how LMs may inadvertently elicit PII from training data. However, to what extent LMs can provide privacy-preserving summaries given a non-private source document remains under-explored. In this paper, we perform a comprehensive study across two closed- and three open-weight LMs of different sizes and families. We experiment with prompting and fine-tuning strategies for privacy-preservation across a range of summarization datasets across three domains. Our extensive quantitative and qualitative analysis including human evaluation shows that LMs often cannot prevent PII leakage on their summaries and that current widely-used metrics cannot capture context dependent privacy risks.

Related papers

• Membership Inference Attack against Long-Context Large Language Models [8.788010048413188]
  We argue that integrating all information into the long context makes it a repository of sensitive information. We propose six membership inference attack strategies tailored for LCLMs. We examine the underlying reasons why LCLMs are susceptible to revealing such membership information.
  arXiv  Detail & Related papers  (2024-11-18T09:50:54Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• LLM-PBE: Assessing Data Privacy in Large Language Models [111.58198436835036]
  Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs.
  arXiv  Detail & Related papers  (2024-08-23T01:37:29Z)
• RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation [73.2390735383842]
  We introduce the first sample-efficient algorithm for LMDPs without any additional structural assumptions. We show how these can be used to derive near-optimal guarantees of an optimistic exploration algorithm. These results can be valuable for a wide range of interactive learning problems beyond LMDPs, and especially, for partially observed environments.
  arXiv  Detail & Related papers  (2024-06-03T14:51:27Z)
• FABLES: Evaluating faithfulness and content selection in book-length summarization [55.50680057160788]
  In this paper, we conduct the first large-scale human evaluation of faithfulness and content selection on book-length documents. We collect FABLES, a dataset of annotations on 3,158 claims made in LLM-generated summaries of 26 books, at a cost of $5.2K USD. An analysis of the annotations reveals that most unfaithful claims relate to events and character states, and they generally require indirect reasoning over the narrative to invalidate.
  arXiv  Detail & Related papers  (2024-04-01T17:33:38Z)
• On Protecting the Data Privacy of Large Language Models (LLMs): A Survey [35.48984524483533]
  Large language models (LLMs) are complex artificial intelligence systems capable of understanding, generating and translating human language. LLMs process and generate large amounts of data, which may threaten data privacy.
  arXiv  Detail & Related papers  (2024-03-08T08:47:48Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey [43.063650238194384]
  Large Language Models (LLMs) have shown greatly enhanced performance in recent years, attributed to increased size and extensive training data. Training data memorization in Machine Learning models scales with model size, particularly concerning for LLMs. Memorized text sequences have the potential to be directly leaked from LLMs, posing a serious threat to data privacy.
  arXiv  Detail & Related papers  (2023-09-27T15:15:23Z)
• Analyzing Leakage of Personally Identifiable Information in Language Models [13.467340359030855]
  Language Models (LMs) have been shown to leak information about training data through sentence-level membership inference and reconstruction attacks. Scrubbing techniques reduce but do not prevent the risk of PII leakage. It is unclear to which extent algorithmic defenses such as differential privacy, designed to guarantee user-level privacy, prevent PII disclosure.
  arXiv  Detail & Related papers  (2023-02-01T16:04:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.