Time-Uniform Self-Normalized Concentration for Discounted Least Squares: Limits and Corrections

• URL: http://arxiv.org/abs/2608.19643v1
• Date: Thu, 20 Aug 2026 05:19:11 GMT
• Title: Time-Uniform Self-Normalized Concentration for Discounted Least Squares: Limits and Corrections
• Abstract summary: Self-normalized concentration inequalities are standard tools in bandit and reinforcement-learning analyses.<n>A widely used weighted extension claims an analogous time-uniform guarantee for discounted least-squares estimators in non-stationary problems.<n>We show that the weighted inequality remains valid at each fixed deterministic time, give valid finite- and infinite-horizon corrections, and discuss consequences for downstream analyses.
• Score: 1.599072005190786
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-normalized concentration inequalities are standard tools in bandit and reinforcement-learning analyses. A widely used weighted extension claims an analogous time-uniform guarantee for discounted least-squares estimators in non-stationary problems. A simple scalar Gaussian counterexample with a fixed parameter shows that the claimed bounded radius is crossed with probability one. For fixed discount and regularization parameters, we further show that, when $δ\leq1/2$ and $T/δ$ is sufficiently large, any deterministic anytime boundary valid uniformly over the stated conditionally sub-Gaussian model class must be at least of order $R\sqrt{\log(T/δ)}$ at some time by horizon $T$; for nondecreasing boundaries, this order is required at time $T$. We identify the proof error: different terminal times use different Gaussian mixing distributions, so the fixed-time mixtures do not form one supermartingale, and the stopping-time argument does not repair this failure. Finally, we show that the weighted inequality remains valid at each fixed deterministic time, give valid finite- and infinite-horizon corrections, and discuss consequences for downstream analyses.

Related papers

• The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives [26.189892200366895]
  We study the time-uniform convergence of the raw iterate of standard descent gradient (SGD) for unconstrained smooth convex objectives.<n>We prove that, under standard noise assumptions, the time-uniform convergence rate gets arbitrarily close to $sqrtlog n / n$ but never reaches it.
  arXiv  Detail & Related papers  (2026-09-08T10:24:19Z)
• Constrained Online Learning with Noisy Constraint Values [55.29259818039367]
  Under common feasibility, our LEDGER algorithm attains $O(sqrt T)$ expected regret and $O(sqrtTlog(eT))$ expected budget violation.<n>Neither a Slater condition, independence between feedback channels, nor an absolute constraint-value bound is needed.
  arXiv  Detail & Related papers  (2026-09-07T01:38:41Z)
• Is Spurious Correlation Removal Always Learnable? [56.28155520961125]
  Invariant learning can fail even when the structure is statistically identifiable.<n>We show a conditional computational barrier: under a black-box samplable supervised sparse recovery primitive, there exist emphamplable multi-dimensional environments.<n>Synthetic and real datasets illustrate the predicted gaps and transitions and motivate simple diversity diagnostics.
  arXiv  Detail & Related papers  (2026-06-11T05:49:43Z)
• Self-Normalized Martingales and Uniform Regret Bounds for Linear Regression [65.82017723631897]
  We show that scale-invariant upper bounds on self-normalized martingales are possible.<n>We derive a self-normalized concentration inequality free of the usual regularization penalties.
  arXiv  Detail & Related papers  (2026-05-02T22:39:00Z)
• Tail allocation for conformal prediction intervals [1.0163716169047106]
  We study split-conformal prediction for regression when the reported prediction set must be a single interval, at target marginal coverage $1-$, where $$ is the nominal miscoverage level.<n>Under this reporting constraint, the natural conditional target is the shortest interval with conditional mass at least $1-$, rather than an equal-tailed interval or a possibly disconnected high-probability set.<n>We parameterize this single-interval by a lower-tail allocation, which determines how the nominal miscoverage $$ is split between the two endpoints.
  arXiv  Detail & Related papers  (2026-04-28T04:14:27Z)
• Quick Change Detection in Discrete-Time in Presence of a Covert Adversary [7.58317340007754]
  We study the problem of covert quickest change detection in a discrete-time setting, where a sequence of observations undergoes a distributional change at an unknown time.<n>We characterize the behavior of the average detection delay (ADD) and the average time to false alarm (AT2FA) when the post-change distribution converges to the pre-change distribution as $to infty$.<n>We identify the critical scaling laws governing covert behavior and derive explicit conditions under which an adversary can maintain covertness, defined by ADD = $()$, whereas in the classical setting, ADD grows only as $
  arXiv  Detail & Related papers  (2026-01-27T19:56:31Z)
• Policy evaluation from a single path: Multi-step methods, mixing and mis-specification [45.88067550131531]
  We study non-parametric estimation of the value function of an infinite-horizon $gamma$-discounted Markov reward process. We provide non-asymptotic guarantees for a general family of kernel-based multi-step temporal difference estimates.
  arXiv  Detail & Related papers  (2022-11-07T23:15:25Z)
• A lower confidence sequence for the changing mean of non-negative right heavy-tailed observations with bounded mean [9.289846887298854]
  A confidence sequence produces an adapted sequence of sets for a predictable parameter sequence with a time-parametric coverage guarantee. This work constructs a non-asymptotic lower CS for the running average conditional expectation whose slack converges to zero.
  arXiv  Detail & Related papers  (2022-10-20T09:50:05Z)
• Role of boundary conditions in the full counting statistics of topological defects after crossing a continuous phase transition [62.997667081978825]
  We analyze the role of boundary conditions in the statistics of topological defects. We show that for fast and moderate quenches, the cumulants of the kink number distribution present a universal scaling with the quench rate.
  arXiv  Detail & Related papers  (2022-07-08T09:55:05Z)
• Counterfactual inference in sequential experiments [17.817769460838665]
  We consider after-study statistical inference for sequentially designed experiments wherein multiple units are assigned treatments for multiple time points.<n>Our goal is to provide inference guarantees for the counterfactual mean at the smallest possible scale.<n>We illustrate our theory via several simulations and a case study involving data from a mobile health clinical trial HeartSteps.
  arXiv  Detail & Related papers  (2022-02-14T17:24:27Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• The Variational Method of Moments [65.91730154730905]
  conditional moment problem is a powerful formulation for describing structural causal parameters in terms of observables. Motivated by a variational minimax reformulation of OWGMM, we define a very general class of estimators for the conditional moment problem. We provide algorithms for valid statistical inference based on the same kind of variational reformulations.
  arXiv  Detail & Related papers  (2020-12-17T07:21:06Z)
• Finite Time Analysis of Linear Two-timescale Stochastic Approximation with Markovian Noise [28.891930079358954]
  We provide a finite-time analysis for linear two timescale SA scheme. Our bounds show that there is no discrepancy in the convergence rate between Markovian and martingale noise. We present an expansion of the expected error with a matching lower bound.
  arXiv  Detail & Related papers  (2020-02-04T13:03:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.