Exact Non-Oblivious Performance of Rademacher Random Embeddings
- URL: http://arxiv.org/abs/2303.11774v1
- Date: Tue, 21 Mar 2023 11:45:27 GMT
- Title: Exact Non-Oblivious Performance of Rademacher Random Embeddings
- Authors: Maciej Skorski and Alessandro Temperoni
- Abstract summary: This paper revisits the performance of Rademacher random projections.
It establishes novel statistical guarantees that are numerically sharp and non-oblivious with respect to the input data.
- Score: 79.28094304325116
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper revisits the performance of Rademacher random projections,
establishing novel statistical guarantees that are numerically sharp and
non-oblivious with respect to the input data. More specifically, the central
result is the Schur-concavity property of Rademacher random projections with
respect to the inputs. This offers a novel geometric perspective on the
performance of random projections, while improving quantitatively on bounds
from previous works. As a corollary of this broader result, we obtained the
improved performance on data which is sparse or is distributed with small
spread. This non-oblivious analysis is a novelty compared to techniques from
previous work, and bridges the frequently observed gap between theory and
practise. The main result uses an algebraic framework for proving
Schur-concavity properties, which is a contribution of independent interest and
an elegant alternative to derivative-based criteria.
Related papers
- von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
We explore a family of expressive and interpretable distributions over circle-valued random functions.
The resulting probability model has connections with continuous spin models in statistical physics.
For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
arXiv Detail & Related papers (2024-06-19T01:57:21Z) - Doubly Robust Inference in Causal Latent Factor Models [12.116813197164047]
This article introduces a new estimator of average treatment effects under unobserved confounding in modern data-rich environments featuring large numbers of units and outcomes.
We derive finite-sample weighting and guarantees, and show that the error of the new estimator converges to a mean-zero Gaussian distribution at a parametric rate.
arXiv Detail & Related papers (2024-02-18T17:13:46Z) - Adaptive Dimension Reduction and Variational Inference for Transductive
Few-Shot Classification [2.922007656878633]
We propose a new clustering method based on Variational Bayesian inference, further improved by Adaptive Dimension Reduction.
Our proposed method significantly improves accuracy in the realistic unbalanced transductive setting on various Few-Shot benchmarks.
arXiv Detail & Related papers (2022-09-18T10:29:02Z) - Robust and Provable Guarantees for Sparse Random Embeddings [72.24615341588846]
We improve upon the guarantees for sparse random embeddings provided by Freksen at al. (NIPS'18) and Jagadeesan (NIPS'18)
We show that (a) our bounds are explicit as opposed to the guarantees provided previously, and (b) our bounds are guaranteed to be sharper by practically significant constants.
We empirically demonstrate that our bounds significantly outperform prior works on a wide range of real-world datasets.
arXiv Detail & Related papers (2022-02-22T11:15:59Z) - Deconfounding Scores: Feature Representations for Causal Effect
Estimation with Weak Overlap [140.98628848491146]
We introduce deconfounding scores, which induce better overlap without biasing the target of estimation.
We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data.
In particular, we show that this technique could be an attractive alternative to standard regularizations.
arXiv Detail & Related papers (2021-04-12T18:50:11Z) - Adversarial Estimation of Riesz Representers [21.510036777607397]
We propose an adversarial framework to estimate the Riesz representer using general function spaces.
We prove a nonasymptotic mean square rate in terms of an abstract quantity called the critical radius, then specialize it for neural networks, random forests, and reproducing kernel Hilbert spaces as leading cases.
arXiv Detail & Related papers (2020-12-30T19:46:57Z) - Sharper convergence bounds of Monte Carlo Rademacher Averages through
Self-Bounding functions [4.518012967046983]
We derive sharper probabilistic concentration bounds for the Monte Carlo Empirical Rademacher Averages.
New results are applicable to yield sharper bounds to (Local) Rademacher Averages.
arXiv Detail & Related papers (2020-10-22T23:05:16Z) - Slice Sampling for General Completely Random Measures [74.24975039689893]
We present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables.
The efficacy of the proposed algorithm is evaluated on several popular nonparametric models.
arXiv Detail & Related papers (2020-06-24T17:53:53Z) - Nonparametric Score Estimators [49.42469547970041]
Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models.
We provide a unifying view of these estimators under the framework of regularized nonparametric regression.
We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
arXiv Detail & Related papers (2020-05-20T15:01:03Z) - Gaussian Process Boosting [13.162429430481982]
We introduce a novel way to combine boosting with Gaussian process and mixed effects models.
We obtain increased prediction accuracy compared to existing approaches on simulated and real-world data sets.
arXiv Detail & Related papers (2020-04-06T13:19:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.