Subsampling for supervised learning in reproducing kernel Hilbert spaces

• URL: http://arxiv.org/abs/2606.21260v1
• Date: Fri, 19 Jun 2026 09:36:10 GMT
• Title: Subsampling for supervised learning in reproducing kernel Hilbert spaces
• Abstract summary: Subsampling aims at reducing the computational cost and time of the estimation step.<n>This work focuses on a nonparametric setting, in which the hypotheses set lies in a reproducing space.<n>By studying the properties of this estimator, we reveal an optimal subsampling scheme and show that it can be used via plug-in.
• Score: 0.40105987447353786
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the era of big data, subsampling became a common practice in statistical learning. By selecting a subgroup of individuals based on which the learner is trained, subsampling aims at reducing the computational cost and time of the estimation step, and ideally leads to a decrease of its energy consumption and carbon footprint. This work focuses on a nonparametric setting, in which the hypotheses set lies in a reproducing kernel Hilbert space, and the estimator is a minimizer of an empirical risk reweighted à la Horvitz-Thompson. By studying the asymptotic properties of this estimator, we reveal an optimal subsampling scheme (regarding the trace of the covariance operator) and show that it can be used via plug-in. A numerical study on synthetic and real-world datasets shows the practicability and the benefit of the proposed approach.

Related papers

• Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regime [17.718748692214508]
  We develop a generative counterpart to the theory of benign overfitting and algorithmic regularization for neural networks.<n>We derive exact risk trajectories under gradient flow training.<n>We analyze how these estimators combine along the reverse-time SDE and characterize the distribution of the resulting samples.
  arXiv  Detail & Related papers  (2026-08-25T00:54:59Z)
• Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach [5.33725915743382]
  Federated learning (FL) has emerged as a powerful paradigm for collaborative model training across distributed clients.<n>Existing FL algorithms predominantly focus on unconstrained optimization problems with exact gradient information.
  arXiv  Detail & Related papers  (2025-07-30T17:24:27Z)
• Prior shift estimation for positive unlabeled data through the lens of kernel embedding [0.3568466510804538]
  We study estimation of a class prior for unlabeled target samples which possibly differs from that of source population.<n>We introduce a novel direct estimator of a class prior which avoids estimation of posterior probabilities in both populations.<n>We show that the proposal works consistently on par or better than its competitors.
  arXiv  Detail & Related papers  (2025-02-28T16:12:53Z)
• Off-policy estimation with adaptively collected data: the power of online learning [20.023469636707635]
  We consider estimation of a linear functional of the treatment effect using adaptively collected data. We propose a general reduction scheme that allows one to produce a sequence of estimates for the treatment effect via online learning.
  arXiv  Detail & Related papers  (2024-11-19T10:18:27Z)
• Symmetric Q-learning: Reducing Skewness of Bellman Error in Online Reinforcement Learning [55.75959755058356]
  In deep reinforcement learning, estimating the value function is essential to evaluate the quality of states and actions. A recent study suggested that the error distribution for training the value function is often skewed because of the properties of the Bellman operator. We proposed a method called Symmetric Q-learning, in which the synthetic noise generated from a zero-mean distribution is added to the target values to generate a Gaussian error distribution.
  arXiv  Detail & Related papers  (2024-03-12T14:49:19Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• A Tale of Sampling and Estimation in Discounted Reinforcement Learning [50.43256303670011]
  We present a minimax lower bound on the discounted mean estimation problem. We show that estimating the mean by directly sampling from the discounted kernel of the Markov process brings compelling statistical properties.
  arXiv  Detail & Related papers  (2023-04-11T09:13:17Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.