PromptPET: Privacy-Utility Optimized Prompt Obfuscation

• URL: http://arxiv.org/abs/2607.02932v1
• Date: Fri, 03 Jul 2026 03:59:34 GMT
• Title: PromptPET: Privacy-Utility Optimized Prompt Obfuscation
• Abstract summary: We aim to protect user privacy via a user-side mechanism that transforms sensitive information in a user prompt.<n>This approach faces an inherent tradeoff between protecting privacy and preserving utility.<n>We introduce PROMPTPET, an agent that selects the best obfuscation action for each sensitive part of the prompt.
• Score: 6.5675177002666665
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Privacy is an important challenge when users interact with AI chatbots, since users may share sensitive information, explicitly or implicitly, and AI chatbots can use this information for user profiling. In this paper, we aim to protect user privacy via a user-side mechanism that transforms sensitive information in a user prompt, while preserving enough information to elicit a useful response from the chatbot. This approach faces an inherent tradeoff between protecting privacy (i.e., avoiding profiling) and preserving utility (i.e., getting personalized and task-specific responses). To that end, we consider, evaluate, and compare four different obfuscation actions, namely redaction, abstraction, replacement, and a novel noising/denoising scheme that we introduce. Additional novel insights include: utilizing a data type taxonomy to both identify and obfuscate sensitive information and explicitly taking into account the utility of chat responses in making the obfuscation decision. First, we systematically optimize and evaluate each obfuscation action independently in terms of the privacy-utility tradeoff it achieves. Second, we propose PROMPTPET, an LLM-based agent that selects the best obfuscation action for each sensitive part of the prompt, using a reinforcement-learning inspired rule optimizer, applied for the first time in this context. Using a real-world chat dataset, we show that PROMPTPET matches the best privacy-utility tradeoff attainable by any single obfuscation action and significantly outperforms prior state-of-the-art approaches.

Related papers

• Investigating In-Context Privacy Learning by Integrating User-Facing Privacy Tools into Conversational Agents [0.1390311627586184]
  We investigate in-context, experiential learning by examining how interactions with privacy tools enhance users' privacy learning.<n>We integrate a just-in-time privacy notice panel into ChatGPT's interface which we simulated.<n>We qualitatively analyzed participants' perceptions of privacy before and after the task sessions and interface design features that supported or hindered user-led protection of sensitive information.
  arXiv  Detail & Related papers  (2026-03-19T19:18:40Z)
• Understanding Users' Privacy Reasoning and Behaviors During Chatbot Use to Support Meaningful Agency in Privacy [0.1390311627586184]
  We examined students' in-the-moment disclosure and protection behaviors, as well as the reasoning underlying these behaviors.<n>Participants used a simulated ChatGPT interface with and without a privacy notice panel that intercepts message submissions.<n>We analyzed how the panel fostered privacy awareness, encouraged protective actions, and supported context-specific reasoning about what information to protect and how.
  arXiv  Detail & Related papers  (2026-01-26T04:13:45Z)
• Semantically-Aware LLM Agent to Enhance Privacy in Conversational AI Services [0.0]
  We present a semantically-aware privacy agent designed to safeguard sensitive PII data when using remote Large Language Models (LLMs)<n>Unlike prior work that often degrade response quality, our approach dynamically replaces sensitive PII entities in user prompts with semantically consistent pseudonyms.<n>Our results show that LOPSIDED reduces semantic utility errors by a factor of 5 compared to baseline techniques.
  arXiv  Detail & Related papers  (2025-10-30T21:34:23Z)
• Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Search [60.197239728279534]
  Large language models (LLMs) in cloud-based services have raised significant privacy concerns.<n>Existing text anonymization and de-identification techniques, such as rule-based redaction and scrubbing, often struggle to balance privacy preservation with text naturalness and utility.<n>We propose a zero-shot, tree-search-based iterative sentence rewriting algorithm that systematically obfuscates or deletes private information while preserving coherence, relevance, and naturalness.
  arXiv  Detail & Related papers  (2025-09-25T07:23:52Z)
• VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks [51.68795949691009]
  We introduce VoxGuard, a framework grounded in differential privacy and membership inference.<n>For attributes, we show that simple transparent attacks recover gender and accent with near-perfect accuracy even after anonymization.<n>Our results demonstrate that EER substantially underestimates leakage, highlighting the need for low-FPR evaluation.
  arXiv  Detail & Related papers  (2025-09-22T20:57:48Z)
• Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents [33.26308626066122]
  We characterize the notion of contextual privacy for user interactions with Conversational Agents (LCAs)<n>It aims to minimize privacy risks by ensuring that users (sender) disclose only information that is both relevant and necessary for achieving their intended goals.<n>We propose a locally deployable framework that operates between users and LCAs, identifying and reformulating out-of-context information in user prompts.
  arXiv  Detail & Related papers  (2025-02-22T09:05:39Z)
• Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy [64.32494202656801]
  Privacy-preserving computer vision is an important emerging problem in machine learning and artificial intelligence.<n>We present anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context.<n>We also proposeMaskDP to protect non-anonymized but privacy sensitive background information.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human [56.46355425175232]
  We suggest sanitizing sensitive text using two common strategies used by humans.<n>We curate the first corpus, coined NAP2, through both crowdsourcing and the use of large language models.<n>Compared to the prior works on anonymization, the human-inspired approaches result in more natural rewrites.
  arXiv  Detail & Related papers  (2024-06-06T05:07:44Z)
• Words Blending Boxes. Obfuscating Queries in Information Retrieval using Differential Privacy [7.831978389504435]
  When an Information Retrieval System (IRS) does not protect the privacy of its users, sensitive information may be disclosed through the queries sent to the system. Recent improvements, especially in NLP, have shown the potential of using Differential Privacy to obfuscate texts. We propose Word Blending Boxes, a novel differentially private mechanism for query obfuscation.
  arXiv  Detail & Related papers  (2024-05-15T12:51:36Z)
• OPOM: Customized Invisible Cloak towards Face Privacy Protection [58.07786010689529]
  We investigate the face privacy protection from a technology standpoint based on a new type of customized cloak. We propose a new method, named one person one mask (OPOM), to generate person-specific (class-wise) universal masks. The effectiveness of the proposed method is evaluated on both common and celebrity datasets.
  arXiv  Detail & Related papers  (2022-05-24T11:29:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.