Concave Statistical Utility Maximization Bandits via Influence-Function Gradients

• URL: http://arxiv.org/abs/2604.22140v3
• Date: Wed, 29 Apr 2026 11:34:22 GMT
• Title: Concave Statistical Utility Maximization Bandits via Influence-Function Gradients
• Abstract summary: We study multi-armed bandits in which the objective is a statistical functional of the long-run reward distribution, rather than expected reward alone.<n>We develop a framework for general concave distributional utilities, with numerical experiments comparing exact and plug-in influence-function implementations.
• Score: 45.88028371034407
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study stochastic multi-armed bandits in which the objective is a statistical functional of the long-run reward distribution, rather than expected reward alone. Under mild continuity assumptions, we show that the infinite-horizon problem reduces to optimizing over stationary mixed policies: each weight vector \(w\) on the simplex induces a mixture law \(P^w\), and performance is measured by the concave utility \(U(w)=\mathfrak U(P^w)\). For differentiable statistical utilities, we use influence-function calculus to derive stochastic gradient estimators from bandit feedback. This leads to an entropic mirror-ascent algorithm on a truncated simplex, implemented through multiplicative-weights updates and plug-in estimates of the influence function. We establish regret bounds that separate the mirror-ascent optimization error from the bias caused by estimating the influence function. The framework is developed for general concave distributional utilities and illustrated through variance and Wasserstein objectives, with numerical experiments comparing exact and plug-in influence-function implementations.

Related papers

• Prediction-Powered Causal Inference by Automatic Debiased Machine Learning and Semi-Supervised Riesz Regression [6.44705221140412]
  This study investigates semiparametric efficient estimation of causal and structural parameters in a semi-supervised setting.<n>In our setting, unlabeled auxiliary regressors are available in addition to labeled observations consisting of outcomes and regressors.
  arXiv  Detail & Related papers  (2026-06-11T04:37:03Z)
• Coarsening Bias from Variable Discretization in Causal Functionals [0.8880611506199766]
  A class of causal effect functionals requires integration over conditional densities of continuous variables.<n>Discretization alters the population-level functional and can induce non-negligible approximation bias.<n>We propose a simple bias-reduced functional that evaluates the outcome regression at within-bin conditional means.
  arXiv  Detail & Related papers  (2026-02-25T16:32:04Z)
• A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage [0.554780083433538]
  We consider supervised learning problems in which set predictions provide explicit uncertainty estimates.<n>We propose a convex loss function for nondecreasing subset-valued functions obtained as level sets of a real-valued function.<n>We show how to naturally obtain sets with optimized conditional coverage.
  arXiv  Detail & Related papers  (2025-12-22T08:41:31Z)
• Malliavin Calculus with Weak Derivatives for Counterfactual Stochastic Optimization [16.179901221618156]
  We study counterfactual optimization of conditional loss functionals under misspecified noisy gradient information.<n>We show using Malliavin calculus that the conditional loss functional of a diffusion process admits an exact representation as a Skorohod integral, yielding variance comparable to classical Monte-Carlo.
  arXiv  Detail & Related papers  (2025-09-30T21:37:54Z)
• Semiparametric conformal prediction [79.6147286161434]
  We construct a conformal prediction set accounting for the joint correlation structure of the vector-valued non-conformity scores.<n>We flexibly estimate the joint cumulative distribution function (CDF) of the scores.<n>Our method yields desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Optimal convex $M$-estimation via score matching [6.115859302936817]
  We construct a data-driven convex loss function with respect to which empirical risk minimisation yields optimal variance in the downstream estimation of the regression coefficients.<n>At the population level, the negative derivative of the optimal convex loss is the best decreasing approximation of the derivative of the log-density of the noise distribution.
  arXiv  Detail & Related papers  (2024-03-25T12:23:19Z)
• Kernel-based off-policy estimation without overlap: Instance optimality beyond semiparametric efficiency [53.90687548731265]
  We study optimal procedures for estimating a linear functional based on observational data. For any convex and symmetric function class $mathcalF$, we derive a non-asymptotic local minimax bound on the mean-squared error.
  arXiv  Detail & Related papers  (2023-01-16T02:57:37Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Wasserstein Distributionally Robust Estimation in High Dimensions: Performance Analysis and Optimal Hyperparameter Tuning [2.4578723416255754]
  Distributionally robust optimization (DRO) has become a powerful framework for estimation under uncertainty.<n>We propose a DRO-based method for linear regression and address a central question: how to optimally choose the robustness radius.<n>We show that our method achieves the same effect as cross-validation, but at a fraction of the computational cost.
  arXiv  Detail & Related papers  (2022-06-27T13:02:59Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Heavy-tailed Streaming Statistical Estimation [58.70341336199497]
  We consider the task of heavy-tailed statistical estimation given streaming $p$ samples. We design a clipped gradient descent and provide an improved analysis under a more nuanced condition on the noise of gradients.
  arXiv  Detail & Related papers  (2021-08-25T21:30:27Z)
• Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methods [8.890430804063705]
  We first present the distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators. The distributional result reflects the trade-off between statistical efficiency and function query complexity.
  arXiv  Detail & Related papers  (2021-02-05T19:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.