Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Models

• URL: http://arxiv.org/abs/2402.02987v2
• Date: Mon, 07 Oct 2024 12:11:58 GMT
• Title: Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Models
• Authors: Junjie Chu, Zeyang Sha, Michael Backes, Yang Zhang,
• Abstract summary: GPT models are increasingly being used for task optimization. In this paper, we introduce a straightforward yet potent Conversation Reconstruction Attack. We present two advanced attacks targeting improved reconstruction of past conversations.
• Score: 20.92843974858305
• License:
• Abstract: Significant advancements have recently been made in large language models represented by GPT models. Users frequently have multi-round private conversations with cloud-hosted GPT models for task optimization. Yet, this operational paradigm introduces additional attack surfaces, particularly in custom GPTs and hijacked chat sessions. In this paper, we introduce a straightforward yet potent Conversation Reconstruction Attack. This attack targets the contents of previous conversations between GPT models and benign users, i.e., the benign users' input contents during their interaction with GPT models. The adversary could induce GPT models to leak such contents by querying them with designed malicious prompts. Our comprehensive examination of privacy risks during the interactions with GPT models under this attack reveals GPT-4's considerable resilience. We present two advanced attacks targeting improved reconstruction of past conversations, demonstrating significant privacy leakage across all models under these advanced techniques. Evaluating various defense mechanisms, we find them ineffective against these attacks. Our findings highlight the ease with which privacy can be compromised in interactions with GPT models, urging the community to safeguard against potential abuses of these models' capabilities.

Related papers

• Imposter.AI: Adversarial Attacks with Hidden Intentions towards Aligned Large Language Models [13.225041704917905]
  This study unveils an attack mechanism that capitalizes on human conversation strategies to extract harmful information from large language models. Unlike conventional methods that target explicit malicious responses, our approach delves deeper into the nature of the information provided in responses.
  arXiv  Detail & Related papers  (2024-07-22T06:04:29Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Membership Inference Attacks and Privacy in Topic Modeling [3.503833571450681]
  We propose an attack against topic models that can confidently identify members of the training data. We propose a framework for private topic modeling that incorporates DP vocabulary selection as a pre-processing step.
  arXiv  Detail & Related papers  (2024-03-07T12:43:42Z)
• Query-Based Adversarial Prompt Generation [67.238873588125]
  We build adversarial examples that cause an aligned language model to emit harmful strings. We validate our attack on GPT-3.5 and OpenAI's safety classifier.
  arXiv  Detail & Related papers  (2024-02-19T18:01:36Z)
• Opening A Pandora's Box: Things You Should Know in the Era of Custom GPTs [27.97654690288698]
  We conduct a comprehensive analysis of the security and privacy issues arising from the custom GPT platform by OpenAI. Our systematic examination categorizes potential attack scenarios into three threat models based on the role of the malicious actor. We identify 26 potential attack vectors, with 19 being partially or fully validated in real-world settings.
  arXiv  Detail & Related papers  (2023-12-31T16:49:12Z)
• InferDPT: Privacy-Preserving Inference for Black-box Large Language Model [66.07752875835506]
  InferDPT is the first practical framework for the privacy-preserving Inference of black-box LLMs. RANTEXT is a novel differential privacy mechanism integrated into the perturbation module of InferDPT.
  arXiv  Detail & Related papers  (2023-10-18T18:00:11Z)
• Unveiling Security, Privacy, and Ethical Concerns of ChatGPT [6.588022305382666]
  ChatGPT uses topic modeling and reinforcement learning to generate natural responses. ChatGPT holds immense promise across various industries, such as customer service, education, mental health treatment, personal productivity, and content creation. This paper focuses on security, privacy, and ethics issues, calling for concerted efforts to ensure the development of secure and ethically sound large language models.
  arXiv  Detail & Related papers  (2023-07-26T13:45:18Z)
• DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models [92.6951708781736]
  This work proposes a comprehensive trustworthiness evaluation for large language models with a focus on GPT-4 and GPT-3.5. We find that GPT models can be easily misled to generate toxic and biased outputs and leak private information. Our work illustrates a comprehensive trustworthiness evaluation of GPT models and sheds light on the trustworthiness gaps.
  arXiv  Detail & Related papers  (2023-06-20T17:24:23Z)
• On Evaluating Adversarial Robustness of Large Vision-Language Models [64.66104342002882]
  We evaluate the robustness of large vision-language models (VLMs) in the most realistic and high-risk setting. In particular, we first craft targeted adversarial examples against pretrained models such as CLIP and BLIP. Black-box queries on these VLMs can further improve the effectiveness of targeted evasion.
  arXiv  Detail & Related papers  (2023-05-26T13:49:44Z)
• Defending against Reconstruction Attacks with R\'enyi Differential Privacy [72.1188520352079]
  Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget. We show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature.
  arXiv  Detail & Related papers  (2022-02-15T18:09:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.