Distilling Human-Aligned Privacy Sensitivity Assessment from Large Language Models

• URL: http://arxiv.org/abs/2603.29497v1
• Date: Tue, 31 Mar 2026 09:40:58 GMT
• Title: Distilling Human-Aligned Privacy Sensitivity Assessment from Large Language Models
• Abstract summary: Recent work has shown that large language models (LLMs) can serve as reliable privacy evaluators, achieving strong agreement with human judgments.<n>We address this gap by distilling the privacy assessment capabilities of Mistral Large 3 (675B) into lightweight encoder models with as few as 150M parameters.
• Score: 7.9049991577473735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate privacy evaluation of textual data remains a critical challenge in privacy-preserving natural language processing. Recent work has shown that large language models (LLMs) can serve as reliable privacy evaluators, achieving strong agreement with human judgments; however, their computational cost and impracticality for processing sensitive data at scale limit real-world deployment. We address this gap by distilling the privacy assessment capabilities of Mistral Large 3 (675B) into lightweight encoder models with as few as 150M parameters. Leveraging a large-scale dataset of privacy-annotated texts spanning 10 diverse domains, we train efficient classifiers that preserve strong agreement with human annotations while dramatically reducing computational requirements. We validate our approach on human-annotated test data and demonstrate its practical utility as an evaluation metric for de-identification systems.

Related papers

• From Automation to Collaboration: Human-in-the-Loop Methods for Safe and Trustworthy NLP [0.28675177318965045]
  Large language models are widely deployed in high-stakes NLP tasks, yet risks as bias, hallucination, adversarial vulnerability and unreliable generalization remain.<n>This survey examines recent human-in-the-loop methods that shift NLP from automation toward collaboration for safety and robustness.
  arXiv  Detail & Related papers  (2026-05-24T19:27:20Z)
• Text-Based Personas for Simulating User Privacy Decisions [23.902474542548944]
  We present Narriva, an approach that generates text-based synthetic privacy personas.<n>We analyze the characteristics of Narriva's synthetic personas in modeling both individual and population-level privacy preferences.
  arXiv  Detail & Related papers  (2026-03-20T09:28:15Z)
• Challenges in Enabling Private Data Valuation [17.450381366291754]
  Data valuation methods quantify how individual training examples contribute to a model's behavior.<n> valuation scores can reveal whether a person's data was included in training, whether it was unusually influential, or what sensitive patterns exist in proprietary datasets.<n>Privacy is fundamentally in tension with valuation utility under differential privacy (DP)
  arXiv  Detail & Related papers  (2026-02-27T22:21:14Z)
• Privasis: Synthesizing the Largest "Public" Private Dataset from Scratch [101.49955223689268]
  We present Privasis, the first million-scale fully synthetic dataset entirely built from scratch.<n>Compared to existing datasets, Privasis offers orders-of-magnitude larger scale with quality.<n>We leverage Privasis to construct a parallel corpus for text sanitization with our pipeline that decomposes texts and applies targeted sanitization.
  arXiv  Detail & Related papers  (2026-02-03T06:54:46Z)
• Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark [10.645183619391135]
  We introduce EAPrivacy, a comprehensive evaluation benchmark designed to quantify the physical-world privacy awareness of embodied agents.<n>EAPrivacy utilizes procedurally generated scenarios across four tiers to test an agent's ability to handle sensitive objects, adapt to changing environments, balance task execution with privacy constraints, and resolve conflicts with social norms.
  arXiv  Detail & Related papers  (2025-09-27T23:39:56Z)
• CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection [60.52240468810558]
  We introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews.<n>We also develop CoCoDet, an AI review detector via a multi-task learning framework, to achieve more accurate and robust detection of AI involvement in review content.
  arXiv  Detail & Related papers  (2025-08-28T06:03:11Z)
• LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Data [47.76073133338117]
  Despite advances in the field of privacy- Natural Language Processing (NLP), the accurate evaluation of privacy remains a challenge.<n>We present a global approach inspired by sox2013$ a model for privacy evaluation in textual data.<n>Our findings pave the way for exploring the feasibility of evaluators as privacy evaluators.
  arXiv  Detail & Related papers  (2025-08-16T20:49:41Z)
• NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Human [56.46355425175232]
  We suggest sanitizing sensitive text using two common strategies used by humans.<n>We curate the first corpus, coined NAP2, through both crowdsourcing and the use of large language models.<n>Compared to the prior works on anonymization, the human-inspired approaches result in more natural rewrites.
  arXiv  Detail & Related papers  (2024-06-06T05:07:44Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• Bring Your Own Data! Self-Supervised Evaluation for Large Language Models [52.15056231665816]
  We propose a framework for self-supervised evaluation of Large Language Models (LLMs) We demonstrate self-supervised evaluation strategies for measuring closed-book knowledge, toxicity, and long-range context dependence. We find strong correlations between self-supervised and human-supervised evaluations.
  arXiv  Detail & Related papers  (2023-06-23T17:59:09Z)
• You Are What You Write: Preserving Privacy in the Era of Large Language Models [2.3431670397288005]
  We present an empirical investigation into the extent of the personal information encoded into pre-trained representations by a range of popular models. We show a positive correlation between the complexity of a model, the amount of data used in pre-training, and data leakage.
  arXiv  Detail & Related papers  (2022-04-20T11:12:53Z)
• Semantics-Preserved Distortion for Personal Privacy Protection in Information Management [65.08939490413037]
  This paper suggests a linguistically-grounded approach to distort texts while maintaining semantic integrity. We present two distinct frameworks for semantic-preserving distortion: a generative approach and a substitutive approach. We also explore privacy protection in a specific medical information management scenario, showing our method effectively limits sensitive data memorization.
  arXiv  Detail & Related papers  (2022-01-04T04:01:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.