Higher-Order Equilibrium Tracking for EM-Compressible Online Estimation

• URL: http://arxiv.org/abs/2605.08864v1
• Date: Sat, 09 May 2026 10:30:55 GMT
• Title: Higher-Order Equilibrium Tracking for EM-Compressible Online Estimation
• Abstract summary: We study online estimation in latent-variable models by recasting the problem as tracking a moving empirical equilibrium.<n>Our framework decomposes the online estimate into the frozen batch equilibrium at the current running statistic.<n>We formalize EM-compressibility and EM-jet$R$-compressibility as the structural conditions that make the equilibrium response and the Newton corrector evaluable.
• Score: 29.20065548055442
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study online estimation in latent-variable models by recasting the problem as tracking a moving empirical equilibrium. Standard online EM and stochastic approximation analyses primarily study convergence toward the population parameter and typically do not isolate the empirical batch optimum from the online tracking error at finite horizon. Our framework decomposes the online estimate into the frozen batch equilibrium at the current running statistic and a tracking lag that captures the algorithm's delay behind this moving target. We prove a batch-to-online transfer theorem: provided $\lVert e_T \rVert_{L^{2}} = o(T^{-1/2})$, the online estimator inherits the batch central limit theorem and the sharp first-order risk constant. Our key observation is that the empirical optimum evolves on a smooth equilibrium manifold indexed by the running statistic. An $m$-th order equilibrium-jet predictor combined with an order-$ν$ frozen corrector yields localized tracking rates $O(T^{-ν(m+1)})$. We formalize EM-compressibility and EM-jet$^R$-compressibility as the structural conditions that make the equilibrium response and the Newton corrector evaluable from a retained streaming statistic. The theory is instantiated in latent linear Gaussian covariance estimation, where the first-order scheme operates on a compressed $d \times d$ statistic with explicit finite-sample risk envelopes and a certified restart rule.

Related papers

• The V-fold jackknife for semiparametric inference: variance estimation, confidence intervals, and simultaneous confidence bands [2.715949869811686]
  bootstrap has been a default tool for statistical inference because of its broad applicability and minimal analytic requirements.<n>We develop the $V$-fold jackknife as a computationally efficient and theoretically justified alternative for semiparametric inference.
  arXiv  Detail & Related papers  (2026-07-24T17:05:54Z)
• Error-Conditioned Neural Solvers [36.51479208528461]
  Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task.<n>Recent hybrid methods promote physical correctness by targeting the PDE residual via gradient descent or Gauss-Newton steps, but inherit the compute cost and instability of the underlying classical gradients.<n>We show, theoretically and empirically, that numerically minimizing the PDE residual can be an unreliable proxy for reconstruction accuracy in ill-conditioned systems.
  arXiv  Detail & Related papers  (2026-06-25T17:56:27Z)
• Symmetric Equilibrium Propagation for Thermodynamic Diffusion Training [0.0]
  We show that a bilinearly-coupled analog substrate can physically realize dynamics at a projected three-to-four orders of magnitude energy advantage over digital inference.<n>We resolve whether the emphtraining loop can be closed on the same substrate without routing gradients through an external digital accelerator.
  arXiv  Detail & Related papers  (2026-04-26T17:05:05Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Why is Normalization Preferred? A Worst-Case Complexity Theory for Stochastically Preconditioned SGD under Heavy-Tailed Noise [17.899443444882888]
  We develop a worst-case complexity theory for inequalityally preconditioned gradient descent (SPSGD)<n>We demonstrate that normalization guarantees convergence to a first-order stationary point at rate $mathcalO(T-fracp-13p-2)$ when problem parameters are known, and $mathcalO(T-fracp-12p)$ when problem parameters are unknown.<n>In contrast, we prove that clipping may fail to converge in the worst case due to the statistical dependence between the preconditioner and the gradient estimates.
  arXiv  Detail & Related papers  (2026-02-13T19:29:17Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• Parametric Mean-Field empirical Bayes in high-dimensional linear regression [8.197187859375694]
  We characterize a sharp phase transition behavior for the Empirical Bayes (vEB) estimator.<n>In the first regime, we show how the estimated prior can be calibrated to enable valid coordinate-wise and delocalized inference.<n>In the second regime, we propose a debiasing technique as a way to improve the performance of the vEB estimator.
  arXiv  Detail & Related papers  (2026-01-23T15:44:01Z)
• Convergence, design and training of continuous-time dropout as a random batch method [0.0]
  We study dropout regularization in continuous-time models through the lens of random-batch methods.<n>We construct an unbiased, well-posed estimator that mimics dropout by sampling neuron batches over time intervals of length $h$.<n>We then specialize in a single-layer neural ODE and validate the theory on classification and flow matching.
  arXiv  Detail & Related papers  (2025-10-15T04:19:01Z)
• Online non-parametric likelihood-ratio estimation by Pearson-divergence functional minimization [55.98760097296213]
  We introduce a new framework for online non-parametric LRE (OLRE) for the setting where pairs of iid observations $(x_t sim p, x'_t sim q)$ are observed over time. We provide theoretical guarantees for the performance of the OLRE method along with empirical validation in synthetic experiments.
  arXiv  Detail & Related papers  (2023-11-03T13:20:11Z)
• Improved Convergence of Score-Based Diffusion Models via Prediction-Correction [15.772322871598085]
  Score-based generative models (SGMs) are powerful tools to sample from complex data distributions. This paper addresses the issue by considering a version of the popular predictor-corrector scheme. We first estimate the final distribution via an inexact Langevin dynamics and then revert the process.
  arXiv  Detail & Related papers  (2023-05-23T15:29:09Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Heavy-tailed Streaming Statistical Estimation [58.70341336199497]
  We consider the task of heavy-tailed statistical estimation given streaming $p$ samples. We design a clipped gradient descent and provide an improved analysis under a more nuanced condition on the noise of gradients.
  arXiv  Detail & Related papers  (2021-08-25T21:30:27Z)
• SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression [68.66245730450915]
  We develop an improved method for debiasing predictions and estimating frequentist uncertainty for practical datasets. Our main contribution is SLOE, an estimator of the signal strength with convergence guarantees that reduces the computation time of estimation and inference by orders of magnitude.
  arXiv  Detail & Related papers  (2021-03-23T17:48:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.