Time-uniform confidence bands for the CDF under nonstationarity

• URL: http://arxiv.org/abs/2302.14248v1
• Date: Tue, 28 Feb 2023 02:14:54 GMT
• Title: Time-uniform confidence bands for the CDF under nonstationarity
• Abstract summary: Estimation of the complete distribution of a random variable is a useful primitive for both manual and automated decision making. We present time-uniform and value-uniform bounds on the CDF of the running averaged conditional distribution of a real-valued random variable. The importance-weighted extension is appropriate for estimating complete counterfactual distributions of rewards given controlled experimentation data exhaust.
• Score: 9.289846887298854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Estimation of the complete distribution of a random variable is a useful primitive for both manual and automated decision making. This problem has received extensive attention in the i.i.d. setting, but the arbitrary data dependent setting remains largely unaddressed. Consistent with known impossibility results, we present computationally felicitous time-uniform and value-uniform bounds on the CDF of the running averaged conditional distribution of a real-valued random variable which are always valid and sometimes trivial, along with an instance-dependent convergence guarantee. The importance-weighted extension is appropriate for estimating complete counterfactual distributions of rewards given controlled experimentation data exhaust, e.g., from an A/B test or a contextual bandit.

Related papers

• Beyond Marginal Validity: Finite-Sample Guarantees for Localized Conformal Prediction [25.59471163637912]
  Conformal prediction endows arbitrary black-box predictors with finite-sample, distribution-free marginal coverage.<n>We provide finite-sample guarantees for the realized localized set that jointly control conditional validity and oracle efficiency.<n>We analyze data-split learned scores: when the score targets a pivotal score, uniform local guarantees decompose into fixed-score calibration and uniform score-estimation errors.
  arXiv  Detail & Related papers  (2026-08-06T15:59:57Z)
• Distributionally Faithful Imputation via Positive Semi-Definite Kernel Density Estimation [55.47411235954028]
  We recast imputation under missing completely at random as density estimation from masked observations.<n>We obtain a convex empirical risk problem with closed form marginals, solvable by a Newton interior point method.<n>The resulting PSD Impute model yields both single and multiple imputations from the same fitted density.
  arXiv  Detail & Related papers  (2026-07-08T15:51:05Z)
• Conformalized Percentile Interval: Finite Sample Validity and Improved Conditional Performance [9.975170291185004]
  We propose a conformal-style calibration method for responses obtained by the probability integral transform (PIT) of the conditional cumulative distribution function (CDF) estimated via neural networks.<n>Experiments on diverse synthetic and real-world benchmarks demonstrate better conditional calibration and substantially shorter intervals than existing methods.
  arXiv  Detail & Related papers  (2026-05-04T23:51:36Z)
• Robust Out-of-Distribution Stochastic Optimization [26.59660885389812]
  We propose a novel data-driven framework that effectively utilizes relevant data distributions for robust decision-making under unseen distributions.<n>We learn a data-driven uncertainty set in reproducing a kernel space (RKHS) from relevant data distributions, with adjustable conservatism.<n>Under distribution generation, we establish rigorous out-of-distribution generalization guarantees for the uncertainty set as well as the solution.
  arXiv  Detail & Related papers  (2026-04-22T03:24:41Z)
• Finite-sample guarantees for data-driven forward-backward operator methods [1.9336815376402718]
  We consider the problem of finding a zero of the sum of two operators, where one is either unavailable in closed form or computationally expensive to evaluate.<n>Under the lens of algorithmic stability, we derive probabilistic bounds on the distance between a true zero and the FB output.<n>We then specialize our results to a popular FB Nash seeking algorithm and validate our theoretical bounds on a control problem for smart grids.
  arXiv  Detail & Related papers  (2025-12-22T09:07:09Z)
• COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guarantees [51.5976496056012]
  COIN is an uncertainty-guarding selection framework that calibrates statistically valid thresholds to filter a single generated answer per question.<n>COIN estimates the empirical error rate on a calibration set and applies confidence interval methods to establish a high-probability upper bound on the true error rate.<n>We demonstrate COIN's robustness in risk control, strong test-time power in retaining admissible answers, and predictive efficiency under limited calibration data.
  arXiv  Detail & Related papers  (2025-06-25T07:04:49Z)
• A Sample Efficient Conditional Independence Test in the Presence of Discretization [54.047334792855345]
  Conditional Independence (CI) tests directly to discretized data can lead to incorrect conclusions.<n>Recent advancements have sought to infer the correct CI relationship between the latent variables through binarizing observed data.<n>Motivated by this, this paper introduces a sample-efficient CI test that does not rely on the binarization process.
  arXiv  Detail & Related papers  (2025-06-10T12:41:26Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional general score-mismatched diffusion samplers. We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification [59.81904428056924]
  We introduce SMASH: a Score MAtching estimator for learning markedPs with uncertainty quantification. Specifically, our framework adopts a normalization-free objective by estimating the pseudolikelihood of markedPs through score-matching. The superior performance of our proposed framework is demonstrated through extensive experiments in both event prediction and uncertainty quantification.
  arXiv  Detail & Related papers  (2023-10-25T02:37:51Z)
• Robust Independence Tests with Finite Sample Guarantees for Synchronous Stochastic Linear Systems [0.0]
  The paper introduces robust independence tests with non-asymptotically guaranteed significance levels for linear time-invariant systems. Our method provides bounds for the type I error probabilities that are distribution-free, i.e., the innovations can have arbitrary distributions.
  arXiv  Detail & Related papers  (2023-08-03T21:13:34Z)
• Probabilistic Learning of Multivariate Time Series with Temporal Irregularity [25.91078012394032]
  temporal irregularities, including nonuniform time intervals and component misalignment. We develop a conditional flow representation to non-parametrically represent the data distribution, which is typically non-Gaussian. The broad applicability and superiority of the proposed solution are confirmed by comparing it with existing approaches through ablation studies and testing on real-world datasets.
  arXiv  Detail & Related papers  (2023-06-15T14:08:48Z)
• Sequential Predictive Two-Sample and Independence Testing [114.4130718687858]
  We study the problems of sequential nonparametric two-sample and independence testing. We build upon the principle of (nonparametric) testing by betting.
  arXiv  Detail & Related papers  (2023-04-29T01:30:33Z)
• Nearest-Neighbor Sampling Based Conditional Independence Testing [15.478671471695794]
  Conditional randomization test (CRT) was recently proposed to test whether two random variables X and Y are conditionally independent given random variables Z. The aim of this paper is to develop a novel alternative of CRT by using nearest-neighbor sampling without assuming the exact form of the distribution of X given Z.
  arXiv  Detail & Related papers  (2023-04-09T07:54:36Z)
• Normalizing Flow Ensembles for Rich Aleatoric and Epistemic Uncertainty Modeling [21.098866735156207]
  We propose an ensemble of Normalizing Flows (NF) which are state-of-the-art in modeling aleatoric uncertainty. The ensembles are created via sets of fixed dropout masks, making them less expensive than creating separate NF models. We demonstrate how to leverage the unique structure of NFs, base distributions, to estimate aleatoric uncertainty without relying on samples.
  arXiv  Detail & Related papers  (2023-02-02T18:38:35Z)
• Conformal Frequency Estimation using Discrete Sketched Data with Coverage for Distinct Queries [35.67445122503686]
  This paper develops conformal inference methods to construct a confidence interval for the frequency of a queried object in a very large discrete data set. We show our methods have improved empirical performance compared to existing frequentist and Bayesian alternatives in simulations.
  arXiv  Detail & Related papers  (2022-11-09T00:05:29Z)
• Data-Driven Approximations of Chance Constrained Programs in Nonstationary Environments [3.126118485851773]
  We study sample average approximations (SAA) of chance constrained programs. We consider a nonstationary variant of this problem, where the random samples are assumed to be independently drawn in a sequential fashion. We propose a novel robust SAA method exploiting information about the Wasserstein distance between the sequence of data-generating distributions and the actual chance constraint distribution.
  arXiv  Detail & Related papers  (2022-05-08T01:01:57Z)
• Decentralized Local Stochastic Extra-Gradient for Variational Inequalities [125.62877849447729]
  We consider distributed variational inequalities (VIs) on domains with the problem data that is heterogeneous (non-IID) and distributed across many devices. We make a very general assumption on the computational network that covers the settings of fully decentralized calculations. We theoretically analyze its convergence rate in the strongly-monotone, monotone, and non-monotone settings.
  arXiv  Detail & Related papers  (2021-06-15T17:45:51Z)
• Batch Stationary Distribution Estimation [98.18201132095066]
  We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. We propose a consistent estimator that is based on recovering a correction ratio function over the given data.
  arXiv  Detail & Related papers  (2020-03-02T09:10:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.