A functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting

• URL: http://arxiv.org/abs/2606.25494v1
• Date: Wed, 24 Jun 2026 07:22:17 GMT
• Title: A functional central limit theorem for kernel gradient flow and infinitesimal gradient boosting
• Abstract summary: We study the fluctuations of the process around its deterministic limit and establish a functional central limit theorem.<n>The proof rests on a general perturbation analysis of ODEs in Banach spaces.
• Score: 1.503974529275767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building on the large-sample analysis of infinitesimal gradient boosting (Dombry and Duchamps, 2024b), we study the fluctuations of the process around its deterministic limit and establish a functional central limit theorem: the rescaled deviations converge in distribution to a Gaussian process. The analysis is carried out in a reproducing kernel Hilbert space (RKHS) naturally associated with the softmax gradient tree base learner, in which the boosting process is characterized as the solution of an autonomous ordinary differential equation (ODE). The proof rests on a general stochastic perturbation analysis of ODEs in Banach spaces, which is of independent interest: whenever a sequence of vector fields converges and satisfies a central limit theorem, so does the associated ODE solution. We first illustrate this perturbation approach in the simpler setting of kernel gradient flow, where the Gaussian limit admits an explicit characterization, and then consider the more complicated tree-based gradient boosting setting.

Related papers

• Dynamic Regret for Online Regression in RKHS via Discounted VAW and Subspace Approximation [0.0]
  We study online regression with the square loss in a kernel reproducing a Hilbert space under a dynamic regret criterion.<n>The proposed method transfers the finite-dimensional discounted Vovk--Azoury--Warmuth approach of Jacobsen & Cutkosky (2024) to the RKHS setting.
  arXiv  Detail & Related papers  (2026-04-27T21:53:54Z)
• Functional Central Limit Theorem for Stochastic Gradient Descent [2.064612766965483]
  We study the shape of the trajectory of the gradient descent algorithm applied to a convex objective function.<n>We prove a functional central limit theorem for the properly rescaled trajectory.
  arXiv  Detail & Related papers  (2026-02-17T12:42:19Z)
• Hellinger-Kantorovich Gradient Flows: Global Exponential Decay of Entropy Functionals [52.154685604660465]
  We investigate a family of gradient flows of positive and probability measures, focusing on the Hellinger-Kantorovich (HK) geometry.<n>A central contribution is a complete characterization of global exponential decay behaviors of entropy functionals under Otto-Wasserstein and Hellinger-type gradient flows.
  arXiv  Detail & Related papers  (2025-01-28T16:17:09Z)
• Limit Theorems for Stochastic Gradient Descent with Infinite Variance [51.4853131023238]
  We show that the gradient descent algorithm can be characterized as the stationary distribution of a suitably defined Ornstein-rnstein process driven by an appropriate L'evy process.<n>We also explore the applications of these results in linear regression and logistic regression models.
  arXiv  Detail & Related papers  (2024-10-21T09:39:10Z)
• Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold [59.73080197971106]
  This paper presents a first-order conjugate optimization method that converges faster than the steepest descent method. It aims to achieve global convergence over the Stiefel manifold.
  arXiv  Detail & Related papers  (2023-08-21T08:02:16Z)
• Last-Iterate Convergence of Adaptive Riemannian Gradient Descent for Equilibrium Computation [52.73824786627612]
  This paper establishes new convergence results for textitgeodesic strongly monotone games.<n>Our key result shows that RGD attains last-iterate linear convergence in a textitgeometry-agnostic fashion.<n>Overall, this paper presents the first geometry-agnostic last-iterate convergence analysis for games beyond the Euclidean settings.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Gradient is All You Need? [0.0]
  In this paper we provide a novel analytical perspective on the theoretical understanding of learning algorithms by interpreting consensus-based gradient-based optimization (CBO) Our results prove the intrinsic power of CBO to alleviate the complexities of the nonlocal landscape function.
  arXiv  Detail & Related papers  (2023-06-16T11:30:55Z)
• Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems [98.34292831923335]
  Motivated by the problem of online correlation analysis, we propose the emphStochastic Scaled-Gradient Descent (SSD) algorithm. We bring these ideas together in an application to online correlation analysis, deriving for the first time an optimal one-time-scale algorithm with an explicit rate of local convergence to normality.
  arXiv  Detail & Related papers  (2021-12-29T18:46:52Z)
• Deep Learning Approximation of Diffeomorphisms via Linear-Control Systems [91.3755431537592]
  We consider a control system of the form $dot x = sum_i=1lF_i(x)u_i$, with linear dependence in the controls. We use the corresponding flow to approximate the action of a diffeomorphism on a compact ensemble of points.
  arXiv  Detail & Related papers  (2021-10-24T08:57:46Z)
• Infinitesimal gradient boosting [0.0]
  We define infinitesimal gradient boosting as a limit of the popular tree-based gradient boosting algorithm from machine learning. We introduce a new class of randomized regression trees bridging totally randomized trees and Extra Trees.
  arXiv  Detail & Related papers  (2021-04-26T15:09:05Z)
• A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions [3.4792548480344254]
  We show that the risk function of the gradient descent method does indeed converge to zero. A key contribution of this work is to explicitly specify a Lyapunov function for the gradient flow system of the ANN parameters.
  arXiv  Detail & Related papers  (2021-02-19T13:33:03Z)
• SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives [86.01677297601624]
  We propose a novel approach for scaling GP regression with derivatives based on quadrature Fourier features. We prove deterministic, non-asymptotic and exponentially fast decaying error bounds which apply for both the approximated kernel as well as the approximated posterior.
  arXiv  Detail & Related papers  (2020-03-05T14:33:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.