Online monotone density estimation and log-optimal calibration

• URL: http://arxiv.org/abs/2602.08927v1
• Date: Mon, 09 Feb 2026 17:29:39 GMT
• Title: Online monotone density estimation and log-optimal calibration
• Abstract summary: We study the problem of online monotone density estimation, where density estimators must be constructed in a predictable manner from sequentially observed data.<n>We propose two online estimators: an online analogue of the classical Grenander estimator, and an expert aggregation estimator inspired by exponential weighting methods from the online learning literature.
• Score: 32.71136783315003
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the problem of online monotone density estimation, where density estimators must be constructed in a predictable manner from sequentially observed data. We propose two online estimators: an online analogue of the classical Grenander estimator, and an expert aggregation estimator inspired by exponential weighting methods from the online learning literature. In the well-specified stochastic setting, where the underlying density is monotone, we show that the expected cumulative log-likelihood gap between the online estimators and the true density admits an $O(n^{1/3})$ bound. We further establish a $\sqrt{n\log{n}}$ pathwise regret bound for the expert aggregation estimator relative to the best offline monotone estimator chosen in hindsight, under minimal regularity assumptions on the observed sequence. As an application of independent interest, we show that the problem of constructing log-optimal p-to-e calibrators for sequential hypothesis testing can be formulated as an online monotone density estimation problem. We adapt the proposed estimators to build empirically adaptive p-to-e calibrators and establish their optimality. Numerical experiments illustrate the theoretical results.

Related papers

• Target-Aware Linear Regression Under Distribution Shift [2.705806052277982]
  Distribution shift between training and deployment is a pervasive challenge for modern AI systems.<n>We study this problem with a stable conditional mean $E[Ymid X]$ across source and target.<n>We identify the hybrid-loss estimator, which jointly incorporates both target marginals, as a benchmark target-aware estimator.
  arXiv  Detail & Related papers  (2026-06-22T02:24:50Z)
• Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity [9.668478511115683]
  We propose a minimum Wasserstein distance estimation framework that avoids explicit modeling of outcome regressions or importance weights.<n>The resulting W-estimator admits a closed-form expression and is numerically equivalent to a classical 1-nearest neighbor estimator.<n> Numerical simulations, along with an analysis of a rainfall dataset, underscore the exceptional performance of our W-estimator.
  arXiv  Detail & Related papers  (2026-01-12T07:36:44Z)
• SCOPE: Spectral Concentration by Distributionally Robust Joint Covariance-Precision Estimation [17.613523305238278]
  We propose a distributionally robust formulation for simultaneously estimating the covariance matrix and the precision matrix of a random vector.<n>We demonstrate that the proposed distributionally robust estimation model can be reduced to a convex optimization problem.<n>We show that our shrinkage estimators perform competitively against state-of-the-art estimators in practical applications.
  arXiv  Detail & Related papers  (2025-11-18T05:13:49Z)
• On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization [57.179679246370114]
  A potential limitation of existing methods is the bias inherent in most perturbation estimators unless a stepsize is proposed.<n>We propose a novel family of unbiased gradient scaling estimators that eliminate bias while maintaining favorable construction.
  arXiv  Detail & Related papers  (2025-10-22T18:25:43Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate.<n>We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Nearest Neighbor Sampling for Covariate Shift Adaptation [7.940293148084844]
  We propose a new covariate shift adaptation method which avoids estimating the weights. The basic idea is to directly work on unlabeled target data, labeled according to the $k$-nearest neighbors in the source dataset. Our experiments show that it achieves drastic reduction in the running time with remarkable accuracy.
  arXiv  Detail & Related papers  (2023-12-15T17:28:09Z)
• Sobolev Space Regularised Pre Density Models [51.558848491038916]
  We propose a new approach to non-parametric density estimation that is based on regularizing a Sobolev norm of the density. This method is statistically consistent, and makes the inductive validation model clear and consistent.
  arXiv  Detail & Related papers  (2023-07-25T18:47:53Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• Faster Wasserstein Distance Estimation with the Sinkhorn Divergence [0.0]
  The squared Wasserstein distance is a quantity to compare probability distributions in a non-parametric setting. In this work, we propose instead to estimate it with the Sinkhorn divergence. We show that, for smooth densities, this estimator has a comparable sample complexity but allows higher regularization levels.
  arXiv  Detail & Related papers  (2020-06-15T06:58:16Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.