Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases

• URL: http://arxiv.org/abs/2506.17336v2
• Date: Tue, 01 Jul 2025 16:41:35 GMT
• Title: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases
• Authors: Yubeen Bae, Minchan Kim, Jaejin Lee, Sangbum Kim, Jaehyung Kim, Yejin Choi, Niloofar Mireshghallah,
• Abstract summary: Large language models (LLMs) are increasingly used as personal agents, accessing sensitive user data such as calendars, emails, and medical records.<n>Users currently face a trade-off: They can send private records to powerful but untrusted LLM providers, increasing their exposure risk.
• Score: 40.94869284214594
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) are increasingly used as personal agents, accessing sensitive user data such as calendars, emails, and medical records. Users currently face a trade-off: They can send private records, many of which are stored in remote databases, to powerful but untrusted LLM providers, increasing their exposure risk. Alternatively, they can run less powerful models locally on trusted devices. We bridge this gap. Our Socratic Chain-of-Thought Reasoning first sends a generic, non-private user query to a powerful, untrusted LLM, which generates a Chain-of-Thought (CoT) prompt and detailed sub-queries without accessing user data. Next, we embed these sub-queries and perform encrypted sub-second semantic search using our Homomorphically Encrypted Vector Database across one million entries of a single user's private data. This represents a realistic scale of personal documents, emails, and records accumulated over years of digital activity. Finally, we feed the CoT prompt and the decrypted records to a local language model and generate the final response. On the LoCoMo long-context QA benchmark, our hybrid framework, combining GPT-4o with a local Llama-3.2-1B model, outperforms using GPT-4o alone by up to 7.1 percentage points. This demonstrates a first step toward systems where tasks are decomposed and split between untrusted strong LLMs and weak local ones, preserving user privacy.

Related papers

• Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences [80.63946798650653]
  We explore how users can stay in control of their data by using privacy profiles.<n>We build a framework where a local model uses these instructions to rewrite queries.<n>To support this research, we introduce a multilingual dataset of real user queries to mark private content.
  arXiv  Detail & Related papers  (2025-07-07T18:22:55Z)
• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• Entailment-Driven Privacy Policy Classification with LLMs [3.564208334473993]
  We propose a framework to classify paragraphs of privacy policies into meaningful labels that are easily understood by users. Our framework improves the F1 score in average by 11.2%.
  arXiv  Detail & Related papers  (2024-09-25T05:07:05Z)
• Evaluating LLM-based Personal Information Extraction and Countermeasures [63.91918057570824]
  Large language model (LLM) based personal information extraction can be benchmarked.<n>LLM can be misused by attackers to accurately extract various personal information from personal profiles.<n> prompt injection can defend against strong LLM-based attacks, reducing the attack to less effective traditional ones.
  arXiv  Detail & Related papers  (2024-08-14T04:49:30Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• PFID: Privacy First Inference Delegation Framework for LLMs [34.59282305562392]
  This paper introduces a novel privacy-preservation framework named PFID for LLMs. It addresses critical privacy concerns by localizing user data through model sharding and singular value decomposition.
  arXiv  Detail & Related papers  (2024-06-18T03:27:09Z)
• PermLLM: Private Inference of Large Language Models within 3 Seconds under WAN [19.014325509263536]
  ChatGPT marks the arrival of the large language model (LLM) era. PermLLM achieves two-party private inference of the ChatGLM-6B model at the speed of around 3s/token.
  arXiv  Detail & Related papers  (2024-05-29T04:06:50Z)
• Protecting Copyrighted Material with Unique Identifiers in Large Language Model Training [55.321010757641524]
  A major public concern regarding the training of large language models (LLMs) is whether they abusing copyrighted online text. Previous membership inference methods may be misled by similar examples in vast amounts of training data. We propose an alternative textitinsert-and-detection methodology, advocating that web users and content platforms employ textbftextitunique identifiers.
  arXiv  Detail & Related papers  (2024-03-23T06:36:32Z)
• CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models [49.60006012946767]
  We propose CodeChameleon, a novel jailbreak framework based on personalized encryption tactics. We conduct extensive experiments on 7 Large Language Models, achieving state-of-the-art average Attack Success Rate (ASR) Remarkably, our method achieves an 86.6% ASR on GPT-4-1106.
  arXiv  Detail & Related papers  (2024-02-26T16:35:59Z)
• EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs [34.77734655124251]
  EmojiPrompt performs generative transformation, obfuscating private data within prompts with linguistic and non-linguistic elements.<n>We evaluate EmojiPrompt's performance across 8 datasets from various domains.<n>EmojiPrompt's atomic-level obfuscation allows it to function exclusively with cloud-based LLMs.
  arXiv  Detail & Related papers  (2024-02-08T17:57:11Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.