Protecting User Prompts Via Character-Level Differential Privacy

• URL: http://arxiv.org/abs/2603.26032v1
• Date: Fri, 27 Mar 2026 03:02:05 GMT
• Title: Protecting User Prompts Via Character-Level Differential Privacy
• Abstract summary: We propose a new method to sanitize user prompts.<n>Our mechanism uses the randomized response mechanism of differential privacy to randomly and independently perturb each character in a word.<n>The restoration will be able to reconstruct non-sensitive words even when they are perturbed due to cues from the context, as well as the fact that these words are often very common.
• Score: 2.986027976506785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) generate responses based on user prompts. Often, these prompts may contain highly sensitive information, including personally identifiable information (PII), which could be exposed to third parties hosting these models. In this work, we propose a new method to sanitize user prompts. Our mechanism uses the randomized response mechanism of differential privacy to randomly and independently perturb each character in a word. The perturbed text is then sent to a remote LLM, which first performs a prompt restoration and subsequently performs the intended downstream task. The idea is that the restoration will be able to reconstruct non-sensitive words even when they are perturbed due to cues from the context, as well as the fact that these words are often very common. On the other hand, perturbation would make reconstruction of sensitive words difficult because they are rare. We experimentally validate our method on two datasets, i2b2/UTHealth and Enron, using two LLMs: Llama-3.1 8B Instruct and GPT-4o mini. We also compare our approach with a word-level differentially private mechanism, and with a rule-based PII redaction baseline, using a unified privacy-utility evaluation. Our results show that sensitive PII tagged in these datasets are reconstructed at a rate close to the theoretical rate of reconstructing completely random words, whereas non-sensitive words are reconstructed at a much higher rate. Our method has the advantage that it can be applied without explicitly identifying sensitive pieces of information in the prompt, while showing a good privacy-utility tradeoff for downstream tasks.

Related papers

• Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration [14.631414891878878]
  Large language models (LLMs) can generate fluent and convincing text at scale.<n>These concerns make robust detection of machine-generated text increasingly necessary.<n>We propose EchoPrompt, a training-free detector based on latent prompt restoration.
  arXiv  Detail & Related papers  (2026-08-06T08:22:19Z)
• When Helpful Context Leaks: Privacy Risks in Domain-Adapted ASR [17.93682540686544]
  SpeechLLMs are increasingly deployed in professional settings where domain customisation is standard practice.<n>A model adapted to recognise domain-specific terminology can be nudged into transcribing a phonetically similar word from its context.<n>We evaluate a prompt-level mitigation strategy and analyse the accuracy-leakage trade-off across customisation approaches.
  arXiv  Detail & Related papers  (2026-05-27T09:30:36Z)
• Towards Low-Latency Tracking of Multiple Speakers With Short-Context Speaker Embeddings [52.985061676464554]
  We propose a Knowledge Distillation based training approach for short context speaker embedding extraction.<n>We leverage the spatial information of the speaker of interest using beamforming to reduce overlap.<n>Results demonstrate that our models are effective at short-context embedding extraction and more robust to overlap.
  arXiv  Detail & Related papers  (2025-08-18T11:32:13Z)
• LLM Hypnosis: Exploiting User Feedback for Unauthorized Knowledge Injection to All Users [50.18141341939909]
  We describe a vulnerability in language models trained with user feedback.<n>A single user can persistently alter LM knowledge and behavior.<n>We show that this attack can be used to insert factual knowledge the model did not previously possess.
  arXiv  Detail & Related papers  (2025-07-03T17:55:40Z)
• Can Large Language Models Really Recognize Your Name? [23.4365383606717]
  We show that modern large language models (LLMs) regularly overlook human names even in short text snippets due to ambiguous contexts.<n>We propose AMBENCH, a benchmark dataset of seemingly ambiguous human names.
  arXiv  Detail & Related papers  (2025-05-20T16:05:05Z)
• Positional Fragility in LLMs: How Offset Effects Reshape Our Understanding of Memorization Risks [22.636913794637437]
  We show that verbatim memorization is most strongly triggered by short prefixes drawn from the beginning of the context window.<n>We show that shifting sensitive data deeper into the context window suppresses both extractable memorization and degeneration.
  arXiv  Detail & Related papers  (2025-05-19T14:28:35Z)
• Pr$εε$mpt: Sanitizing Sensitive Prompts for LLMs [49.84954577111077]
  Pr$epsilonepsilon$mpt is a novel system that implements a prompt sanitizer.<n>We show that Pr$epsilonepsilon$mpt is a practical method to achieve meaningful privacy guarantees.
  arXiv  Detail & Related papers  (2025-04-07T14:52:40Z)
• Protecting Copyrighted Material with Unique Identifiers in Large Language Model Training [55.321010757641524]
  A primary concern regarding training large language models (LLMs) is whether they abuse copyrighted online text.<n>We propose an alternative textitinsert-and-detect methodology, advocating that web users and content platforms employ textbftextitunique identifiers for reliable and independent membership inference.
  arXiv  Detail & Related papers  (2024-03-23T06:36:32Z)
• Wiki-En-ASR-Adapt: Large-scale synthetic dataset for English ASR Customization [66.22007368434633]
  We present a first large-scale public synthetic dataset for contextual spellchecking customization of automatic speech recognition (ASR) The proposed approach allows creating millions of realistic examples of corrupted ASR hypotheses and simulate non-trivial biasing lists for the customization task. We report experiments with training an open-source customization model on the proposed dataset and show that the injection of hard negative biasing phrases decreases WER and the number of false alarms.
  arXiv  Detail & Related papers  (2023-09-29T14:18:59Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.