CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignment

• URL: http://arxiv.org/abs/2608.09164v2
• Date: Thu, 13 Aug 2026 06:14:13 GMT
• Title: CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignment
• Abstract summary: We introduce CIDER, a dataset of 14,850 human annotations from 169 users.<n>We form 1,650 contextual disclosure boundary sets across 60 interpersonal communication scenarios.<n>In-context personalization improves prediction accuracy by up to 11.41 percentage points using only 6 historical examples.
• Score: 5.186812525968559
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Aligning large language models (LLMs) with human privacy preferences requires capturing individuals' disclosure boundaries beyond general privacy norms. However, a gap remains in eliciting such nuanced preferences to evaluate alignment in realistic settings. We introduce CIDER, a dataset of 14,850 human annotations from 169 users, forming 1,650 contextual disclosure boundary sets across 60 interpersonal communication scenarios involving information sharing that violates privacy norms. Each boundary represents a real user's disclosure decisions over 9 sharing variants in a scenario, for a given communication role and AI-mediated condition. We formulate a task in which models predict a user's disclosure decision from historical boundaries, with varying levels of contextual information. Across 12 open and proprietary models, in-context personalization improves prediction accuracy by up to 11.41 percentage points using only 6 historical examples. Larger models such as GPT-5.4 (with medium reasoning effort) and Claude Sonnet 4.6 are better at leveraging semantic context to understand user-specific, context-dependent disclosure preferences for more accurate predictions, while smaller models tend to rely on structured heuristics based on disclosure granularity and identifiability. Personalization generally improves prediction accuracy, but the improvement is often accompanied by imbalanced shifts in false-positive and false-negative rates across models, with only Claude Sonnet 4.6 achieving balanced improvements in both. Our findings reveal both the promise and limitations of inference-time personalization for privacy preference modeling and position CIDER as a resource for advancing personalized privacy alignment.

Related papers

• Evaluating the Hidden Costs of Personalization in Large Language Models [55.07935163900691]
  We identify three emerging risks: (1) irrelevant personalization, where models reference personal information in unnecessary contexts; (2) preference narrowing, where models reinforce informational echo chambers; and (3) sycophantic bias, where models agree excessively with user opinions.<n>We propose PRISK, a dynamic evaluation framework with automated data generation and tailored metrics that uncovers systematic limitations in current AI assistants.
  arXiv  Detail & Related papers  (2026-08-28T20:05:12Z)
• Private Direct Preference Optimization for LLM Alignment [77.56374641532715]
  Direct preference optimization (DPO) is now a standard method for aligning large language models (LLMs) using human preference data.<n>In this paper, we formalize preference privacy, a label-DP-style privacy notion for DPO.<n>We then design PrivDPO, a DPO variant that enforces preference privacy while remaining compatible with large-scale LLM training.
  arXiv  Detail & Related papers  (2026-08-05T16:51:02Z)
• Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modeling [2.805986764620217]
  We propose a privacy-preserving framework that imposes differential privacy only on reward learning.<n>We show that privacy contributes an additional additive term beyond the usual non-private statistical error.<n>Experiments on the Anthropic HH-RLHF dataset show stronger private alignment performance than existing differentially private baseline methods.
  arXiv  Detail & Related papers  (2026-03-23T20:45:17Z)
• Value-Action Alignment in Large Language Models under Privacy-Prosocial Conflict [5.739904853130176]
  We introduce a context-based assessment protocol that administers standardized questionnaires for privacy attitudes, prosocialness, and acceptance of data sharing.<n>To evaluate value-action alignments under competing attitudes, we use multi-group structural equation modeling (MGSEM) to identify relations from privacy concerns and prosocialness to data sharing.
  arXiv  Detail & Related papers  (2026-01-07T03:30:42Z)
• Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark [10.645183619391135]
  We introduce EAPrivacy, a comprehensive evaluation benchmark designed to quantify the physical-world privacy awareness of embodied agents.<n>EAPrivacy utilizes procedurally generated scenarios across four tiers to test an agent's ability to handle sensitive objects, adapt to changing environments, balance task execution with privacy constraints, and resolve conflicts with social norms.
  arXiv  Detail & Related papers  (2025-09-27T23:39:56Z)
• Personalized Language Models via Privacy-Preserving Evolutionary Model Merging [53.97323896430374]
  Personalization in language models aims to tailor model behavior to individual users or user groups.<n>We propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME)<n>PriME employs gradient-free methods to directly optimize utility while reducing privacy risks.<n>Experiments on the LaMP benchmark show that PriME consistently outperforms a range of baselines, achieving up to a 45% improvement in task performance.
  arXiv  Detail & Related papers  (2025-03-23T09:46:07Z)
• Optimal Private Discrete Distribution Estimation with One-bit Communication [63.413106413939836]
  We consider a private discrete distribution estimation problem with one-bit communication constraint. We characterize the first-orders of the worst-case trade-off under the one-bit communication constraint. These results demonstrate the optimal dependence of the privacy-utility trade-off under the one-bit communication constraint.
  arXiv  Detail & Related papers  (2023-10-17T05:21:19Z)
• Learning Personalized Alignment for Evaluating Open-ended Text Generation [44.565686959174585]
  PerSE is an interpretable evaluation framework designed to assess alignment with specific human preferences. It is tuned to infer specific preferences from an in-context personal profile and evaluate the alignment between the generated content and personal preferences. Our 13B LLaMA-2-based PerSE shows a 15.8% increase in Kendall correlation and a 13.7% rise in accuracy with zero-shot reviewers.
  arXiv  Detail & Related papers  (2023-10-05T04:15:48Z)
• Mixed Differential Privacy in Computer Vision [133.68363478737058]
  AdaMix is an adaptive differentially private algorithm for training deep neural network classifiers using both private and public image data. A few-shot or even zero-shot learning baseline that ignores private data can outperform fine-tuning on a large private dataset.
  arXiv  Detail & Related papers  (2022-03-22T06:15:43Z)
• Private Prediction Sets [72.75711776601973]
  Machine learning systems need reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. We evaluate the method on large-scale computer vision datasets.
  arXiv  Detail & Related papers  (2021-02-11T18:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.