Privacy-Preserving Conformal Prediction Under Local Differential Privacy

• URL: http://arxiv.org/abs/2505.15721v1
• Date: Wed, 21 May 2025 16:29:44 GMT
• Title: Privacy-Preserving Conformal Prediction Under Local Differential Privacy
• Abstract summary: We propose two complementary approaches under local differential privacy (LDP)<n>In the first approach, users do not access the model but instead provide their input features and a perturbed label using a k-ary randomized response.<n>In the second approach, which enforces stricter privacy constraints, users add noise to their conformity score by binary search response.
• Score: 19.02072419946821
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conformal prediction (CP) provides sets of candidate classes with a guaranteed probability of containing the true class. However, it typically relies on a calibration set with clean labels. We address privacy-sensitive scenarios where the aggregator is untrusted and can only access a perturbed version of the true labels. We propose two complementary approaches under local differential privacy (LDP). In the first approach, users do not access the model but instead provide their input features and a perturbed label using a k-ary randomized response. In the second approach, which enforces stricter privacy constraints, users add noise to their conformity score by binary search response. This method requires access to the classification model but preserves both data and label privacy. Both approaches compute the conformal threshold directly from noisy data without accessing the true labels. We prove finite-sample coverage guarantees and demonstrate robust coverage even under severe randomization. This approach unifies strong local privacy with predictive uncertainty control, making it well-suited for sensitive applications such as medical imaging or large language model queries, regardless of whether users can (or are willing to) compute their own scores.

Related papers

• Confidence Calibration of Deep Learning Systems [4.907205095294475]
  This thesis develops methods for improving calibration under label noise and domain shifts.<n>We propose a framework that provides valid uncertainty quantification while maintaining privacy guarantees.<n>Our results bridge calibration theory and practical deployment in safety-critical applications.
  arXiv  Detail & Related papers  (2026-08-12T14:23:14Z)
• Judging with Confidence: Calibrating Autoraters to Preference Distributions [56.17041629492863]
  We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population.<n>We present two learning methods tailored to different data conditions.<n>Our results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution.
  arXiv  Detail & Related papers  (2025-09-30T20:36:41Z)
• Benchmarking Fraud Detectors on Private Graph Data [70.4654745317714]
  Currently, many types of fraud are managed in part by automated detection algorithms that operate over graphs.<n>We consider the scenario where a data holder wishes to outsource development of fraud detectors to third parties.<n>Third parties submit their fraud detectors to the data holder, who evaluates these algorithms on a private dataset and then publicly communicates the results.<n>We propose a realistic privacy attack on this system that allows an adversary to de-anonymize individuals' data based only on the evaluation results.
  arXiv  Detail & Related papers  (2025-07-30T03:20:15Z)
• Sparse Activations as Conformal Predictors [19.298282860984116]
  We find a novel connection between conformal prediction and sparse softmax-like transformations.<n>We introduce new non-conformity scores for classification that make the calibration process correspond to the widely used temperature scaling method.<n>We show that the proposed method achieves competitive results in terms of coverage, efficiency, and adaptiveness.
  arXiv  Detail & Related papers  (2025-02-20T17:53:41Z)
• Estimating the Conformal Prediction Threshold from Noisy Labels [22.841631892273547]
  We show how we can estimate the noise-free conformal threshold based on the noisy labeled data.<n>We dub our approach Noise-Aware Conformal Prediction (NACP) and show on several natural and medical image classification datasets.
  arXiv  Detail & Related papers  (2025-01-22T09:35:58Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Enhanced Privacy Bound for Shuffle Model with Personalized Privacy [32.08637708405314]
  Differential Privacy (DP) is an enhanced privacy protocol which introduces an intermediate trusted server between local users and a central data curator. It significantly amplifies the central DP guarantee by anonymizing and shuffling the local randomized data. This work focuses on deriving the central privacy bound for a more practical setting where personalized local privacy is required by each user.
  arXiv  Detail & Related papers  (2024-07-25T16:11:56Z)
• Robust Yet Efficient Conformal Prediction Sets [53.78604391939934]
  Conformal prediction (CP) can convert any model's output into prediction sets guaranteed to include the true label. We derive provably robust sets by bounding the worst-case change in conformity scores.
  arXiv  Detail & Related papers  (2024-07-12T10:59:44Z)
• On Rate-Optimal Partitioning Classification from Observable and from Privatised Data [0.0]
  We revisit the classical method of partitioning classification and study its convergence rate under relaxed conditions. The privacy constraints mean that the data $(X_i$), dots,(X_n,Y_n)$ cannot be directly observed. We add Laplace distributed noises to the discontinuations of all possible locations of the feature vector $X_i$ and to its label $Y_i$.
  arXiv  Detail & Related papers  (2023-12-22T18:07:18Z)
• PAC Prediction Sets Under Label Shift [52.30074177997787]
  Prediction sets capture uncertainty by predicting sets of labels rather than individual labels. We propose a novel algorithm for constructing prediction sets with PAC guarantees in the label shift setting. We evaluate our approach on five datasets.
  arXiv  Detail & Related papers  (2023-10-19T17:57:57Z)
• Practical Adversarial Multivalid Conformal Prediction [27.179891682629183]
  We give a generic conformal prediction method for sequential prediction. It achieves target empirical coverage guarantees against adversarially chosen data. It is computationally lightweight -- comparable to split conformal prediction.
  arXiv  Detail & Related papers  (2022-06-02T14:33:00Z)
• Gaussian Processes with Differential Privacy [3.934224774675743]
  We add strong privacy protection to Gaussian processes (GPs) via differential privacy (DP) We achieve this by using sparse GP methodology and publishing a private variational approximation on known inducing points. Our experiments demonstrate that given sufficient amount of data, the method can produce accurate models under strong privacy protection.
  arXiv  Detail & Related papers  (2021-06-01T13:23:16Z)
• Distribution-free uncertainty quantification for classification under label shift [105.27463615756733]
  We focus on uncertainty quantification (UQ) for classification problems via two avenues. We first argue that label shift hurts UQ, by showing degradation in coverage and calibration. We examine these techniques theoretically in a distribution-free framework and demonstrate their excellent practical performance.
  arXiv  Detail & Related papers  (2021-03-04T20:51:03Z)
• Private Prediction Sets [72.75711776601973]
  Machine learning systems need reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. We evaluate the method on large-scale computer vision datasets.
  arXiv  Detail & Related papers  (2021-02-11T18:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.