Improving the statistical efficiency of cross-conformal prediction

• URL: http://arxiv.org/abs/2503.01495v1
• Date: Mon, 03 Mar 2025 13:07:28 GMT
• Title: Improving the statistical efficiency of cross-conformal prediction
• Authors: Matteo Gasparin, Aaditya Ramdas,
• Abstract summary: Vovk introduced cross-conformal prediction, a modification of split conformal designed to improve the width of prediction sets.<n>We propose new variants of both methods that yield smaller prediction sets without compromising the latter theoretical guarantee.
• Score: 29.43493007296859
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Vovk (2015) introduced cross-conformal prediction, a modification of split conformal designed to improve the width of prediction sets. The method, when trained with a miscoverage rate equal to $\alpha$ and $n \gg K$, ensures a marginal coverage of at least $1 - 2\alpha - 2(1-\alpha)(K-1)/(n+K)$, where $n$ is the number of observations and $K$ denotes the number of folds. A simple modification of the method achieves coverage of at least $1-2\alpha$. In this work, we propose new variants of both methods that yield smaller prediction sets without compromising the latter theoretical guarantee. The proposed methods are based on recent results deriving more statistically efficient combination of p-values that leverage exchangeability and randomization. Simulations confirm the theoretical findings and bring out some important tradeoffs.

Related papers

• Robust Conformal Prediction with a Single Binary Certificate [58.450154976190795]
  Conformal prediction (CP) converts any model's output to prediction sets with a guarantee to cover the true label with (adjustable) high probability. We propose a robust conformal prediction that produces smaller sets even with significantly lower MC samples.
  arXiv  Detail & Related papers  (2025-03-07T08:41:53Z)
• Efficiently Solving Discounted MDPs with Predictions on Transition Matrices [6.199300239433395]
  We study Discounted Markov Decision Processes (DMDPs) under a generative model. We propose a novel framework to investigate how a prediction on the transition matrix can enhance the sample efficiency in solving DMDPs.
  arXiv  Detail & Related papers  (2025-02-21T09:59:46Z)
• Smoothed Normalization for Efficient Distributed Private Optimization [54.197255548244705]
  Federated learning enables machine learning models with privacy of participants. There is no differentially private distributed method for training, non-feedback problems. We introduce a new distributed algorithm $alpha$-$sf NormEC$ with provable convergence guarantees.
  arXiv  Detail & Related papers  (2025-02-19T07:10:32Z)
• Wasserstein-regularized Conformal Prediction under General Distribution Shift [21.29977879431466]
  Conformal prediction yields a prediction set with guaranteed $1-alpha$ coverage of the true target under the i.i.d. assumption.<n>We propose a Wasserstein distance-based upper bound of the coverage gap and analyze the bound using probability measure pushforwards.<n>We exploit the separation to design an algorithm based on importance weighting and regularized representation learning (WR-CP) to reduce the Wasserstein bound with a finite-sample error bound.
  arXiv  Detail & Related papers  (2025-01-23T07:29:44Z)
• SPLITZ: Certifiable Robustness via Split Lipschitz Randomized Smoothing [8.471466670802817]
  SPLITZ is a practical and novel approach to provide certifiable robustness to adversarial examples.<n> Motivation for SPLITZ comes from the observation that many standard deep networks exhibit heterogeneity in Lipschitz constants.<n>We show that SPLITZ consistently improves on existing state-of-the-art approaches in the MNIST, CIFAR-10 and ImageNet datasets.
  arXiv  Detail & Related papers  (2024-07-03T05:13:28Z)
• Verifiably Robust Conformal Prediction [1.391198481393699]
  This paper introduces VRCP (Verifiably Robust Conformal Prediction), a new framework that leverages neural network verification methods to recover coverage guarantees under adversarial attacks. Our method is the first to support perturbations bounded by arbitrary norms including $ell1$, $ell2$, and $ellinfty$, as well as regression tasks. In every case, VRCP achieves above nominal coverage and yields significantly more efficient and informative prediction regions than the SotA.
  arXiv  Detail & Related papers  (2024-05-29T09:50:43Z)
• Provably Robust Conformal Prediction with Improved Efficiency [29.70455766394585]
  Conformal prediction is a powerful tool to generate uncertainty sets with guaranteed coverage. adversarial examples are able to manipulate conformal methods to construct prediction sets with invalid coverage rates. We propose two novel methods, Post-Training Transformation (PTT) and Robust Conformal Training (RCT), to effectively reduce prediction set size with little overhead.
  arXiv  Detail & Related papers  (2024-04-30T15:49:01Z)
• FABind: Fast and Accurate Protein-Ligand Binding [127.7790493202716]
  $mathbfFABind$ is an end-to-end model that combines pocket prediction and docking to achieve accurate and fast protein-ligand binding. Our proposed model demonstrates strong advantages in terms of effectiveness and efficiency compared to existing methods.
  arXiv  Detail & Related papers  (2023-10-10T16:39:47Z)
• Online Multivalid Learning: Means, Moments, and Prediction Intervals [16.75129633574157]
  We present a technique for providing contextual predictions that are "multivalid" in various senses. The resulting estimates correctly predict various statistics of the labels $y$ not just marginally. Because our algorithms handle adversarially chosen examples, they can equally well be used to predict statistics of the residuals of arbitrary point prediction methods.
  arXiv  Detail & Related papers  (2021-01-05T19:08:11Z)
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations [78.23408201652984]
  Top-k predictions are used in many real-world applications such as machine learning as a service, recommender systems, and web searches. Our work is based on randomized smoothing, which builds a provably robust classifier via randomizing an input. For instance, our method can build a classifier that achieves a certified top-3 accuracy of 69.2% on ImageNet when an attacker can arbitrarily perturb 5 pixels of a testing image.
  arXiv  Detail & Related papers  (2020-11-15T21:34:44Z)
• Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification [54.22421582955454]
  We provide the first result of the optimal minimax guarantees for the excess risk for adversarially robust classification. Results are stated in terms of the Adversarial Signal-to-Noise Ratio (AdvSNR), which generalizes a similar notion for standard linear classification to the adversarial setting.
  arXiv  Detail & Related papers  (2020-06-29T21:06:52Z)
• Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework [60.981406394238434]
  We propose a general framework of adversarial certification with non-Gaussian noise and for more general types of attacks. Our proposed methods achieve better certification results than previous works and provide a new perspective on randomized smoothing certification.
  arXiv  Detail & Related papers  (2020-02-21T07:52:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.