Whistledown: Combining User-Level Privacy with Conversational Coherence in LLMs

• URL: http://arxiv.org/abs/2511.13319v1
• Date: Mon, 17 Nov 2025 12:56:33 GMT
• Title: Whistledown: Combining User-Level Privacy with Conversational Coherence in LLMs
• Abstract summary: We present Whistledown, a best-effort privacy layer that modifies prompts before they are sent to the large language models (LLMs)<n>Whistledown combines pseudonymization and $$-local differential privacy ($$-LDP) with transformation caching to provide best-effort privacy protection without sacrificing conversational utility.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Users increasingly rely on large language models (LLMs) for personal, emotionally charged, and socially sensitive conversations. However, prompts sent to cloud-hosted models can contain personally identifiable information (PII) that users do not want logged, retained, or leaked. We observe this to be especially acute when users discuss friends, coworkers, or adversaries, i.e., when they spill the tea. Enterprises face the same challenge when they want to use LLMs for internal communication and decision-making. In this whitepaper, we present Whistledown, a best-effort privacy layer that modifies prompts before they are sent to the LLM. Whistledown combines pseudonymization and $ε$-local differential privacy ($ε$-LDP) with transformation caching to provide best-effort privacy protection without sacrificing conversational utility. Whistledown is designed to have low compute and memory overhead, allowing it to be deployed directly on a client's device in the case of individual users. For enterprise users, Whistledown is deployed centrally within a zero-trust gateway that runs on an enterprise's trusted infrastructure. Whistledown requires no changes to the existing APIs of popular LLM providers.

Related papers

• Guarding Your Conversations: Privacy Gatekeepers for Secure Interactions with Cloud-Based AI Models [0.34998703934432673]
  We propose the concept of an "LLM gatekeeper", a lightweight, locally run model that filters out sensitive information from user queries before they are sent to the potentially untrustworthy, though highly capable, cloud-based LLM.<n>Through experiments with human subjects, we demonstrate that this dual-model approach introduces minimal overhead while significantly enhancing user privacy, without compromising the quality of LLM responses.
  arXiv  Detail & Related papers  (2025-08-22T19:49:03Z)
• Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences [73.5779077857545]
  We build a framework where a local model uses these instructions to rewrite queries, only hiding details deemed sensitive by the user, before sending them to an external model.<n>Experiments with lightweight local LLMs show that, after fine-tuning, they markedly exceed the performance of much larger zero-shot models.<n>At the same time, the system still faces challenges in fully adhering to user instructions, underscoring the need for models with a better understanding of user-defined privacy preferences.
  arXiv  Detail & Related papers  (2025-07-07T18:22:55Z)
• DP-Fusion: Token-Level Differentially Private Inference for Large Language Models [51.71591819896191]
  Large language models (LLMs) do not preserve privacy at inference-time.<n> DP-Fusion provably bounds the influence a set of tokens in the context can have on the LLM's output.<n>We show that our method creates token-level provably privatized documents with substantially improved theoretical and empirical privacy.
  arXiv  Detail & Related papers  (2025-07-06T20:49:39Z)
• A General Pseudonymization Framework for Cloud-Based LLMs: Replacing Privacy Information in Controlled Text Generation [0.6699777383856287]
  ChatGPT services leverage cloud-based large language models (LLMs)<n>Privacy concerns arise as prompts are transmitted and processed by the model providers.<n>We propose a general pseudonymization framework applicable to cloud-based LLMs.
  arXiv  Detail & Related papers  (2025-02-21T06:15:53Z)
• PRIV-QA: Privacy-Preserving Question Answering for Cloud Large Language Models [10.050972891318324]
  We propose a privacy preservation pipeline for protecting privacy and sensitive information during interactions between users and large language models.<n>We construct SensitiveQA, the first privacy open-ended question-answering dataset.<n>Our proposed solution employs a multi-stage strategy aimed at preemptively securing user information while simultaneously preserving the response quality of cloud-based LLMs.
  arXiv  Detail & Related papers  (2025-02-19T09:17:07Z)
• KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server [48.04903443425111]
  Large language models (LLMs) facilitate many parties to fine-tune LLMs on their own private data. Existing solutions, such as utilizing synthetic data for substitution, struggle to simultaneously improve performance and preserve privacy. We propose KnowledgeSG, a novel client-server framework which enhances synthetic data quality and improves model performance while ensuring privacy.
  arXiv  Detail & Related papers  (2024-10-08T06:42:28Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• PFID: Privacy First Inference Delegation Framework for LLMs [34.59282305562392]
  This paper introduces a novel privacy-preservation framework named PFID for LLMs. It addresses critical privacy concerns by localizing user data through model sharding and singular value decomposition.
  arXiv  Detail & Related papers  (2024-06-18T03:27:09Z)
• ConfusionPrompt: Practical Private Inference for Online Large Language Models [3.8134804426693094]
  State-of-the-art large language models (LLMs) are typically deployed as online services, requiring users to transmit detailed prompts to cloud servers. We introduce ConfusionPrompt, a novel framework for private LLM inference that protects user privacy by decomposing the original prompt into smaller sub-prompts. We show that ConfusionPrompt achieves significantly higher utility than local inference methods using open-source models and perturbation-based techniques.
  arXiv  Detail & Related papers  (2023-12-30T01:26:42Z)
• DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer [57.04801796205638]
  Large Language Models (LLMs) have emerged as dominant tools for various tasks. However, concerns surrounding data privacy present obstacles due to the tuned prompts' dependency on sensitive private information. We present Differentially-Private Offsite Prompt Tuning (DP-OPT) to address this challenge.
  arXiv  Detail & Related papers  (2023-11-27T02:01:10Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.