Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents

• URL: http://arxiv.org/abs/2502.18509v1
• Date: Sat, 22 Feb 2025 09:05:39 GMT
• Title: Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents
• Authors: Ivoline Ngong, Swanand Kadhe, Hao Wang, Keerthiram Murugesan, Justin D. Weisz, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy,
• Abstract summary: We characterize the notion of contextual privacy for user interactions with LLMs.<n>It aims to minimize privacy risks by ensuring that users (sender) disclose only information that is both relevant and necessary for achieving their intended goals.
• Score: 33.26308626066122
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Conversational agents are increasingly woven into individuals' personal lives, yet users often underestimate the privacy risks involved. The moment users share information with these agents (e.g., LLMs), their private information becomes vulnerable to exposure. In this paper, we characterize the notion of contextual privacy for user interactions with LLMs. It aims to minimize privacy risks by ensuring that users (sender) disclose only information that is both relevant and necessary for achieving their intended goals when interacting with LLMs (untrusted receivers). Through a formative design user study, we observe how even "privacy-conscious" users inadvertently reveal sensitive information through indirect disclosures. Based on insights from this study, we propose a locally-deployable framework that operates between users and LLMs, and identifies and reformulates out-of-context information in user prompts. Our evaluation using examples from ShareGPT shows that lightweight models can effectively implement this framework, achieving strong gains in contextual privacy while preserving the user's intended interaction goals through different approaches to classify information relevant to the intended goals.

Related papers

• PRIV-QA: Privacy-Preserving Question Answering for Cloud Large Language Models [10.050972891318324]
  We propose a privacy preservation pipeline for protecting privacy and sensitive information during interactions between users and large language models. We construct SensitiveQA, the first privacy open-ended question-answering dataset. Our proposed solution employs a multi-stage strategy aimed at preemptively securing user information while simultaneously preserving the response quality of cloud-based LLMs.
  arXiv  Detail & Related papers  (2025-02-19T09:17:07Z)
• Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbots [2.2447085410328103]
  Rescriber is a browser extension that supports user-led data minimization in LLM-based conversational agents.<n>Our studies showed that Rescriber helped users reduce unnecessary disclosure and addressed their privacy concerns.<n>Our findings confirm the viability of smaller-LLM-powered, user-facing, on-device privacy controls.
  arXiv  Detail & Related papers  (2024-10-10T01:23:16Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Operationalizing Contextual Integrity in Privacy-Conscious Assistants [34.70330533067581]
  We propose to operationalize contextual integrity (CI) to steer advanced AI assistants to behave in accordance with privacy expectations. In particular, we design and evaluate a number of strategies to steer assistants' information-sharing actions to be CI compliant. Our evaluation is based on a novel form filling benchmark composed of human annotations of common webform applications.
  arXiv  Detail & Related papers  (2024-08-05T10:53:51Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• No Free Lunch Theorem for Privacy-Preserving LLM Inference [30.554456047738295]
  This study develops a framework for inferring privacy-protected Large Language Models (LLMs) It lays down a solid theoretical basis for examining the interplay between privacy preservation and utility.
  arXiv  Detail & Related papers  (2024-05-31T08:22:53Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• "It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents [27.480959048351973]
  The widespread use of Large Language Model (LLM)-based conversational agents (CAs) raises many privacy concerns. We analyzed sensitive disclosures in real-world ChatGPT conversations and conducted semi-structured interviews with 19 LLM-based CA users. We found that users are constantly faced with trade-offs between privacy, utility, and convenience when using LLM-based CAs.
  arXiv  Detail & Related papers  (2023-09-20T21:34:36Z)
• Privacy Explanations - A Means to End-User Trust [64.7066037969487]
  We looked into how explainability might help to tackle this problem. We created privacy explanations that aim to help to clarify to end users why and for what purposes specific data is required. Our findings reveal that privacy explanations can be an important step towards increasing trust in software systems.
  arXiv  Detail & Related papers  (2022-10-18T09:30:37Z)
• PCAL: A Privacy-preserving Intelligent Credit Risk Modeling Framework Based on Adversarial Learning [111.19576084222345]
  This paper proposes a framework of Privacy-preserving Credit risk modeling based on Adversarial Learning (PCAL) PCAL aims to mask the private information inside the original dataset, while maintaining the important utility information for the target prediction task performance. Results indicate that PCAL can learn an effective, privacy-free representation from user data, providing a solid foundation towards privacy-preserving machine learning for credit risk analysis.
  arXiv  Detail & Related papers  (2020-10-06T07:04:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.