Multi-Distribution Robust Conformal Prediction

• URL: http://arxiv.org/abs/2601.02998v1
• Date: Tue, 06 Jan 2026 13:22:13 GMT
• Title: Multi-Distribution Robust Conformal Prediction
• Authors: Yuqi Yang, Ying Jin,
• Abstract summary: We study the problem of constructing a conformal prediction set that is uniformly valid across multiple, heterogeneous distributions.<n>We first propose a max-p aggregation scheme that delivers finite-sample, multi-distribution coverage.<n>We discuss how our framework relates to group-wise distributionally robust optimization, sub-population shift, fairness, and multi-source learning.
• Score: 15.5300376981723
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In many fairness and distribution robustness problems, one has access to labeled data from multiple source distributions yet the test data may come from an arbitrary member or a mixture of them. We study the problem of constructing a conformal prediction set that is uniformly valid across multiple, heterogeneous distributions, in the sense that no matter which distribution the test point is from, the coverage of the prediction set is guaranteed to exceed a pre-specified level. We first propose a max-p aggregation scheme that delivers finite-sample, multi-distribution coverage given any conformity scores associated with each distribution. Upon studying several efficiency optimization programs subject to uniform coverage, we prove the optimality and tightness of our aggregation scheme, and propose a general algorithm to learn conformity scores that lead to efficient prediction sets after the aggregation under standard conditions. We discuss how our framework relates to group-wise distributionally robust optimization, sub-population shift, fairness, and multi-source learning. In synthetic and real-data experiments, our method delivers valid worst-case coverage across multiple distributions while greatly reducing the set size compared with naively applying max-p aggregation to single-source conformity scores, and can be comparable in size to single-source prediction sets with popular, standard conformity scores.

Related papers

• Distribution-informed Online Conformal Prediction [53.674678995825666]
  We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule.<n>COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate.<n>We prove that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.
  arXiv  Detail & Related papers  (2025-12-08T17:51:49Z)
• Provably Minimum-Length Conformal Prediction Sets for Ordinal Classification [17.822517655243427]
  We propose an ordinal-CP method that is model-agnostic and provides instance-level optimal prediction intervals.<n>Experiments on four benchmark datasets from diverse domains are conducted to demonstrate the significantly improved predictive efficiency.
  arXiv  Detail & Related papers  (2025-11-20T23:00:15Z)
• Aggregating Conformal Prediction Sets via α-Allocation [22.63151097928744]
  This work introduces a principled aggregation strategy that optimally allocates confidence levels across multiple conformal prediction sets.<n>Experiments on synthetic and real-world datasets demonstrate that COLA achieves considerably smaller prediction sets than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-11-15T07:11:21Z)
• Class conditional conformal prediction for multiple inputs by p-value aggregation [11.198836025239963]
  We introduce an innovative refinement to conformal prediction methods for classification tasks.<n>Our approach is motivated by applications in citizen science, where multiple images of the same plant or animal are captured by individuals.<n>We evaluate our method on simulated and real data, with a particular focus on Pl@ntNet, a prominent citizen science platform.
  arXiv  Detail & Related papers  (2025-07-09T09:17:17Z)
• Direct Distributional Optimization for Provable Alignment of Diffusion Models [39.048284342436666]
  We introduce a novel alignment method for diffusion models from distribution optimization perspectives.<n>We first formulate the problem as a generic regularized loss minimization over probability distributions.<n>We enable sampling from the learned distribution by approximating its score function via Doob's $h$-transform technique.
  arXiv  Detail & Related papers  (2025-02-05T07:35:15Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers.<n>We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.<n>This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Posterior Conformal Prediction [11.576645675675096]
  Conformal prediction is a popular technique for constructing prediction intervals with distribution-free coverage guarantees. This article introduces a new method, posterior conformal prediction (PCP), which generates prediction intervals with both marginal and approximate conditional validity. We evaluate the performance of PCP on diverse datasets from socio-economic, scientific and healthcare applications.
  arXiv  Detail & Related papers  (2024-09-29T14:09:07Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers [143.6249073384419]
  In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers. We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art. In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods.
  arXiv  Detail & Related papers  (2024-01-29T02:08:40Z)
• A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
  Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms. We develop a general framework from the perspective of Bregman minimization divergence. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
  arXiv  Detail & Related papers  (2021-12-07T01:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.