Synthetic-Powered Predictive Inference

• URL: http://arxiv.org/abs/2505.13432v2
• Date: Mon, 16 Jun 2025 14:53:55 GMT
• Title: Synthetic-Powered Predictive Inference
• Authors: Meshi Bashari, Roy Maor Lotan, Yonghoon Lee, Edgar Dobriban, Yaniv Romano,
• Abstract summary: Synthetic-powered predictive inference (SPI)<n>An empirical quantile mapping that aligns nonconformity scores from trusted, real data with those from synthetic data.<n> Experiments on image classification -- augmenting data with synthetic diffusion-model generated images -- demonstrate notable improvements in predictive efficiency in data-scarce settings.
• Score: 28.99972786873634
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conformal prediction is a framework for predictive inference with a distribution-free, finite-sample guarantee. However, it tends to provide uninformative prediction sets when calibration data are scarce. This paper introduces Synthetic-powered predictive inference (SPI), a novel framework that incorporates synthetic data -- e.g., from a generative model -- to improve sample efficiency. At the core of our method is a score transporter: an empirical quantile mapping that aligns nonconformity scores from trusted, real data with those from synthetic data. By carefully integrating the score transporter into the calibration process, SPI provably achieves finite-sample coverage guarantees without making any assumptions about the real and synthetic data distributions. When the score distributions are well aligned, SPI yields substantially tighter and more informative prediction sets than standard conformal prediction. Experiments on image classification -- augmenting data with synthetic diffusion-model generated images -- and on tabular regression demonstrate notable improvements in predictive efficiency in data-scarce settings.

Related papers

• Statistical Guarantees in Synthetic Data through Conformal Adversarial Generation [1.3654846342364308]
  Existing generative models produce compelling synthetic samples but lack rigorous statistical guarantees about their relation to the underlying data distribution.<n>We present a novel framework that incorporates conformal prediction methodologies into Generative Adrial Networks (GANs)<n>This approach, termed Conformalized GAN (cGAN), demonstrates enhanced calibration properties while maintaining the generative power of traditional GANs.
  arXiv  Detail & Related papers  (2025-04-23T19:07:44Z)
• Conditional Data Synthesis Augmentation [4.3108820946281945]
  Conditional Data Synthesis Augmentation (CoDSA) is a novel framework that synthesizes high-fidelity data for improving model performance across multimodal domains.<n>CoDSA fine-tunes pre-trained generative models to enhance the realism of synthetic data and increase sample density in sparse areas.<n>We introduce a theoretical framework that quantifies the statistical accuracy improvements enabled by CoDSA as a function of synthetic sample volume and targeted region allocation.
  arXiv  Detail & Related papers  (2025-04-10T03:38:11Z)
• Noise-Adaptive Conformal Classification with Marginal Coverage [53.74125453366155]
  We introduce an adaptive conformal inference method capable of efficiently handling deviations from exchangeability caused by random label noise.<n>We validate our method through extensive numerical experiments demonstrating its effectiveness on synthetic and real data sets.
  arXiv  Detail & Related papers  (2025-01-29T23:55:23Z)
• Generative Conformal Prediction with Vectorized Non-Conformity Scores [6.059745771017814]
  Conformal prediction provides model-agnostic uncertainty quantification with guaranteed coverage.<n>We propose a generative conformal prediction framework with vectorized non-conformity scores.<n>We construct adaptive uncertainty sets using density-ranked uncertainty balls.
  arXiv  Detail & Related papers  (2024-10-17T16:37:03Z)
• Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning [53.42244686183879]
  Conformal prediction provides model-agnostic and distribution-free uncertainty quantification.<n>Yet, conformal prediction is not reliable under poisoning attacks where adversaries manipulate both training and calibration data.<n>We propose reliable prediction sets (RPS): the first efficient method for constructing conformal prediction sets with provable reliability guarantees under poisoning.
  arXiv  Detail & Related papers  (2024-10-13T15:37:11Z)
• Split Conformal Prediction under Data Contamination [14.23965125128232]
  We study the robustness of split conformal prediction in a data contamination setting. We quantify the impact of corrupted data on the coverage and efficiency of the constructed sets. We propose an adjustment in the classification setting which we call Contamination Robust Conformal Prediction.
  arXiv  Detail & Related papers  (2024-07-10T14:33:28Z)
• Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation [62.2436697657307]
  Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data.<n>We propose a method called Stratified Prediction-Powered Inference (StratPPI)<n>We show that the basic PPI estimates can be considerably improved by employing simple data stratification strategies.
  arXiv  Detail & Related papers  (2024-06-06T17:37:39Z)
• PPI++: Efficient Prediction-Powered Inference [31.403415618169433]
  We present PPI++: a methodology for estimation and inference based on a small labeled dataset and a typically much larger dataset of machine-learning predictions. The methods automatically adapt to the quality of available predictions, yielding easy-to-compute confidence sets. PPI++ builds on prediction-powered inference (PPI), which targets the same problem setting, improving its computational and statistical efficiency.
  arXiv  Detail & Related papers  (2023-11-02T17:59:04Z)
• Perturbation-Assisted Sample Synthesis: A Novel Approach for Uncertainty Quantification [3.175239447683357]
  This paper introduces a novel Perturbation-Assisted Inference (PAI) framework utilizing synthetic data generated by the Perturbation-Assisted Sample Synthesis (PASS) method. The framework focuses on uncertainty quantification in complex data scenarios, particularly involving unstructured data. We demonstrate the effectiveness of PAI in advancing uncertainty quantification in complex, data-driven tasks by applying it to diverse areas such as image synthesis, sentiment word analysis, multimodal inference, and the construction of prediction intervals.
  arXiv  Detail & Related papers  (2023-05-30T01:01:36Z)
• SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data [78.21197488065177]
  Recent success in fine-tuning large models, that are pretrained on broad data at scale, on downstream tasks has led to a significant paradigm shift in deep learning. This paper proposes a new task-agnostic framework, textitSynBench, to measure the quality of pretrained representations using synthetic data.
  arXiv  Detail & Related papers  (2022-10-06T15:25:00Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Balance-Subsampled Stable Prediction [55.13512328954456]
  We propose a novel balance-subsampled stable prediction (BSSP) algorithm based on the theory of fractional factorial design. A design-theoretic analysis shows that the proposed method can reduce the confounding effects among predictors induced by the distribution shift. Numerical experiments on both synthetic and real-world data sets demonstrate that our BSSP algorithm significantly outperforms the baseline methods for stable prediction across unknown test data.
  arXiv  Detail & Related papers  (2020-06-08T07:01:38Z)
• One Step to Efficient Synthetic Data [9.3000873953175]
  A common approach to synthetic data is to sample from a fitted model. We show that this approach results in a sample with inefficient estimators and whose joint distribution is inconsistent with the true distribution. Motivated by this, we propose a general method of producing synthetic data.
  arXiv  Detail & Related papers  (2020-06-03T17:12:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.