Singleton-Optimized Conformal Prediction

• URL: http://arxiv.org/abs/2509.24095v1
• Date: Sun, 28 Sep 2025 22:20:40 GMT
• Title: Singleton-Optimized Conformal Prediction
• Authors: Tao Wang, Yan Sun, Edgar Dobriban,
• Abstract summary: Conformal prediction can be used to construct single prediction sets that cover the true outcome with a desired probability.<n>We propose a new non-oriented score that aims to minimize unambiguous.<n>average probability of producing non-conformity sets.
• Score: 20.966568503308213
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conformal prediction can be used to construct prediction sets that cover the true outcome with a desired probability, but can sometimes lead to large prediction sets that are costly in practice. The most useful outcome is a singleton prediction-an unambiguous decision-yet existing efficiency-oriented methods primarily optimize average set size. Motivated by this, we propose a new nonconformity score that aims to minimize the probability of producing non-singleton sets. Starting from a non-convex constrained optimization problem as a motivation, we provide a geometric reformulation and associated algorithm for computing the nonconformity score and associated split conformal prediction sets in O(K) time for K-class problems. Using this score in split conformal prediction leads to our proposed Singleton-Optimized Conformal Prediction (SOCOP) method. We evaluate our method in experiments on image classification and LLM multiple-choice question-answering, comparing with standard nonconformity scores such as the (negative) label probability estimates and their cumulative distribution function; both of which are motivated by optimizing length. The results show that SOCOP increases singleton frequency (sometimes by over 20%) compared to the above scores, with minimal impact on average set size.

Related papers

• Provably Minimum-Length Conformal Prediction Sets for Ordinal Classification [17.822517655243427]
  We propose an ordinal-CP method that is model-agnostic and provides instance-level optimal prediction intervals.<n>Experiments on four benchmark datasets from diverse domains are conducted to demonstrate the significantly improved predictive efficiency.
  arXiv  Detail & Related papers  (2025-11-20T23:00:15Z)
• Minimum Volume Conformal Sets for Multivariate Regression [44.99833362998488]
  Conformal prediction provides a principled framework for constructing predictive sets with finite-sample validity.<n>We propose an optimization-driven framework based on a novel loss function that directly learns minimum-conformity covering sets.<n>Our approach optimize over prediction sets defined by arbitrary norm balls, including single and multi-norm formulations.
  arXiv  Detail & Related papers  (2025-03-24T18:54:22Z)
• Volume Optimality in Conformal Prediction with Structured Prediction Sets [22.923139209762788]
  Conformal Prediction is a widely studied technique to construct prediction sets of future observations.<n>We first prove an impossibility of volume optimality where any distribution-free method can only find a trivial solution.<n>We then introduce a new notion of volume optimality by restricting the prediction sets to belong to a set family.
  arXiv  Detail & Related papers  (2025-02-23T17:31:33Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Semiparametric conformal prediction [79.6147286161434]
  We construct a conformal prediction set accounting for the joint correlation structure of the vector-valued non-conformity scores.<n>We flexibly estimate the joint cumulative distribution function (CDF) of the scores.<n>Our method yields desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Generative Conformal Prediction with Vectorized Non-Conformity Scores [6.059745771017814]
  Conformal prediction provides model-agnostic uncertainty quantification with guaranteed coverage.<n>We propose a generative conformal prediction framework with vectorized non-conformity scores.<n>We construct adaptive uncertainty sets using density-ranked uncertainty balls.
  arXiv  Detail & Related papers  (2024-10-17T16:37:03Z)
• Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering [55.15192437680943]
  Generative models lack rigorous statistical guarantees for their outputs.<n>We propose a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee.<n>This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example.
  arXiv  Detail & Related papers  (2024-10-02T15:26:52Z)
• Post-Selection Confidence Bounds for Prediction Performance [2.28438857884398]
  In machine learning, the selection of a promising model from a potentially large number of competing models and the assessment of its generalization performance are critical tasks. We propose an algorithm how to compute valid lower confidence bounds for multiple models that have been selected based on their prediction performances in the evaluation set.
  arXiv  Detail & Related papers  (2022-10-24T13:28:43Z)
• Boost Test-Time Performance with Closed-Loop Inference [85.43516360332646]
  We propose to predict hard-classified test samples in a looped manner to boost the model performance. We first devise a filtering criterion to identify those hard-classified test samples that need additional inference loops. For each hard sample, we construct an additional auxiliary learning task based on its original top-$K$ predictions to calibrate the model.
  arXiv  Detail & Related papers  (2022-03-21T10:20:21Z)
• Efficient and Differentiable Conformal Prediction with General Function Classes [96.74055810115456]
  We propose a generalization of conformal prediction to multiple learnable parameters. We show that it achieves approximate valid population coverage and near-optimal efficiency within class. Experiments show that our algorithm is able to learn valid prediction sets and improve the efficiency significantly.
  arXiv  Detail & Related papers  (2022-02-22T18:37:23Z)
• Deep Probabilistic Graph Matching [72.6690550634166]
  We propose a deep learning-based graph matching framework that works for the original QAP without compromising on the matching constraints. The proposed method is evaluated on three popularly tested benchmarks (Pascal VOC, Willow Object and SPair-71k) and it outperforms all previous state-of-the-arts on all benchmarks.
  arXiv  Detail & Related papers  (2022-01-05T13:37:27Z)
• Training conformal predictors [0.0]
  Efficiency criteria for conformal prediction, such as emphobserved fuzziness, are commonly used to emphevaluate the performance of given conformal predictors. Here, we investigate whether it is possible to exploit such criteria to emphlearn classifiers.
  arXiv  Detail & Related papers  (2020-05-14T14:47:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.