Off-the-grid learning of sparse mixtures from a continuous dictionary

• URL: http://arxiv.org/abs/2207.00171v1
• Date: Wed, 29 Jun 2022 07:55:20 GMT
• Title: Off-the-grid learning of sparse mixtures from a continuous dictionary
• Authors: Cristina Butucea (CREST), Jean-Fran\c{c}ois Delmas (CERMICS), Anne Dutfoy (EDF R&D), Cl\'ement Hardy (CERMICS, EDF R&D)
• Abstract summary: We consider a general non-linear model where the signal is a finite mixture of unknown, possibly increasing, number of features issued from a continuous dictionary parameterized by a real nonlinear parameter. We propose an off-the-grid optimization method, that is, a method which does not use any discretization scheme on the parameter space. We use recent results on the geometry of off-the-grid methods to give minimal separation on the true underlying non-linear parameters such that interpolating certificate functions can be constructed.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider a general non-linear model where the signal is a finite mixture of an unknown, possibly increasing, number of features issued from a continuous dictionary parameterized by a real nonlinear parameter. The signal is observed with Gaussian (possibly correlated) noise in either a continuous or a discrete setup. We propose an off-the-grid optimization method, that is, a method which does not use any discretization scheme on the parameter space, to estimate both the non-linear parameters of the features and the linear parameters of the mixture. We use recent results on the geometry of off-the-grid methods to give minimal separation on the true underlying non-linear parameters such that interpolating certificate functions can be constructed. Using also tail bounds for suprema of Gaussian processes we bound the prediction error with high probability. Assuming that the certificate functions can be constructed, our prediction error bound is up to log --factors similar to the rates attained by the Lasso predictor in the linear regression model. We also establish convergence rates that quantify with high probability the quality of estimation for both the linear and the non-linear parameters.

Related papers

• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Overparameterized Multiple Linear Regression as Hyper-Curve Fitting [0.0]
  It is proven that a linear model will produce exact predictions even in the presence of nonlinear dependencies that violate the model assumptions. The hyper-curve approach is especially suited for the regularization of problems with noise in predictor variables and can be used to remove noisy and "improper" predictors from the model.
  arXiv  Detail & Related papers  (2024-04-11T15:43:11Z)
• Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels [78.6096486885658]
  We introduce lower bounds to the linearized Laplace approximation of the marginal likelihood. These bounds are amenable togradient-based optimization and allow to trade off estimation accuracy against computational complexity.
  arXiv  Detail & Related papers  (2023-06-06T19:02:57Z)
• Kernel-based off-policy estimation without overlap: Instance optimality beyond semiparametric efficiency [53.90687548731265]
  We study optimal procedures for estimating a linear functional based on observational data. For any convex and symmetric function class $mathcalF$, we derive a non-asymptotic local minimax bound on the mean-squared error.
  arXiv  Detail & Related papers  (2023-01-16T02:57:37Z)
• Off-the-grid prediction and testing for linear combination of translated features [2.774897240515734]
  We consider a model where a signal (discrete or continuous) is observed with an additive Gaussian noise process. We extend previous prediction results for off-the-grid estimators by taking into account that the scale parameter may vary. We propose a procedure to test whether the features of the observed signal belong to a given finite collection.
  arXiv  Detail & Related papers  (2022-12-02T13:48:45Z)
• Simultaneous off-the-grid learning of mixtures issued from a continuous dictionary [0.4369058206183195]
  We observe a continuum of signals corrupted by noise. Each signal is a finite mixture of an unknown number of features belonging to a continuous dictionary. We formulate regularized optimization problem to estimate simultaneously the linear coefficients in the mixtures.
  arXiv  Detail & Related papers  (2022-10-27T06:43:37Z)
• Gaussian Process Uniform Error Bounds with Unknown Hyperparameters for Safety-Critical Applications [71.23286211775084]
  We introduce robust Gaussian process uniform error bounds in settings with unknown hyper parameters. Our approach computes a confidence region in the space of hyper parameters, which enables us to obtain a probabilistic upper bound for the model error. Experiments show that the bound performs significantly better than vanilla and fully Bayesian processes.
  arXiv  Detail & Related papers  (2021-09-06T17:10:01Z)
• Boosting in Univariate Nonparametric Maximum Likelihood Estimation [5.770800671793959]
  Nonparametric maximum likelihood estimation is intended to infer the unknown density distribution. To alleviate the over parameterization in nonparametric data fitting, smoothing assumptions are usually merged into the estimation. We deduce the boosting algorithm by the second-order approximation of nonparametric log-likelihood.
  arXiv  Detail & Related papers  (2021-01-21T08:46:33Z)
• Understanding Implicit Regularization in Over-Parameterized Single Index Model [55.41685740015095]
  We design regularization-free algorithms for the high-dimensional single index model. We provide theoretical guarantees for the induced implicit regularization phenomenon.
  arXiv  Detail & Related papers  (2020-07-16T13:27:47Z)
• Implicit differentiation of Lasso-type models for hyperparameter optimization [82.73138686390514]
  We introduce an efficient implicit differentiation algorithm, without matrix inversion, tailored for Lasso-type problems. Our approach scales to high-dimensional data by leveraging the sparsity of the solutions.
  arXiv  Detail & Related papers  (2020-02-20T18:43:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.