Robust Bayesian Subspace Identification for Small Data Sets

• URL: http://arxiv.org/abs/2212.14132v1
• Date: Thu, 29 Dec 2022 00:29:04 GMT
• Title: Robust Bayesian Subspace Identification for Small Data Sets
• Authors: Alexandre Rodrigues Mesquita
• Abstract summary: We propose regularized estimators, shrinkage estimators and Bayesian estimation to reduce the effect of variance. Our experimental results show that our proposed estimators may reduce the estimation risk up to $40%$ of that of traditional subspace methods.
• Score: 91.3755431537592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model estimates obtained from traditional subspace identification methods may be subject to significant variance. This elevated variance is aggravated in the cases of large models or of a limited sample size. Common solutions to reduce the effect of variance are regularized estimators, shrinkage estimators and Bayesian estimation. In the current work we investigate the latter two solutions, which have not yet been applied to subspace identification. Our experimental results show that our proposed estimators may reduce the estimation risk up to $40\%$ of that of traditional subspace methods.

Related papers

• One step closer to unbiased aleatoric uncertainty estimation [71.55174353766289]
  We propose a new estimation method by actively de-noising the observed data. By conducting a broad range of experiments, we demonstrate that our proposed approach provides a much closer approximation to the actual data uncertainty than the standard method.
  arXiv  Detail & Related papers  (2023-12-16T14:59:11Z)
• Correcting sampling biases via importance reweighting for spatial modeling [2.6862667248315386]
  In machine learning models, the estimation of errors is often complex due to distribution bias. We introduce an approach based on the ideas of importance sampling to obtain an unbiased estimate of the target error.
  arXiv  Detail & Related papers  (2023-09-09T15:36:28Z)
• Differentially Private Estimation via Statistical Depth [0.0]
  Two notions of statistical depth are used to motivate new approximate DP location and regression estimators. To avoid requiring that users specify a priori bounds on the estimates and/or the observations, variants of these DP mechanisms are described.
  arXiv  Detail & Related papers  (2022-07-26T01:59:07Z)
• Uncertainty Estimation for Heatmap-based Landmark Localization [4.673063715963989]
  We propose Quantile Binning, a data-driven method to categorise predictions by uncertainty with estimated error bounds. We demonstrate this framework by comparing and contrasting three uncertainty measures. We conclude by illustrating how filtering out gross mispredictions caught in our Quantile Bins significantly improves the proportion of predictions under an acceptable error threshold.
  arXiv  Detail & Related papers  (2022-03-04T14:40:44Z)
• On Variance Estimation of Random Forests [0.0]
  This paper develops an unbiased variance estimator based on incomplete U-statistics. We show that our estimators enjoy lower bias and more accurate confidence interval coverage without additional computational costs.
  arXiv  Detail & Related papers  (2022-02-18T03:35:47Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• The Aleatoric Uncertainty Estimation Using a Separate Formulation with Virtual Residuals [51.71066839337174]
  Existing methods can quantify the error in the target estimation, but they tend to underestimate it. We propose a new separable formulation for the estimation of a signal and of its uncertainty, avoiding the effect of overfitting. We demonstrate that the proposed method outperforms a state-of-the-art technique for signal and uncertainty estimation.
  arXiv  Detail & Related papers  (2020-11-03T12:11:27Z)
• Optimal Off-Policy Evaluation from Multiple Logging Policies [77.62012545592233]
  We study off-policy evaluation from multiple logging policies, each generating a dataset of fixed size, i.e., stratified sampling. We find the OPE estimator for multiple loggers with minimum variance for any instance, i.e., the efficient one.
  arXiv  Detail & Related papers  (2020-10-21T13:43:48Z)
• Telescoping Density-Ratio Estimation [21.514983459970903]
  We introduce a new framework, telescoping density-ratio estimation (TRE) TRE enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces. Our experiments demonstrate that TRE can yield substantial improvements over existing single-ratio methods.
  arXiv  Detail & Related papers  (2020-06-22T12:55:06Z)
• Nonparametric Estimation of the Fisher Information and Its Applications [82.00720226775964]
  This paper considers the problem of estimation of the Fisher information for location from a random sample of size $n$. An estimator proposed by Bhattacharya is revisited and improved convergence rates are derived. A new estimator, termed a clipped estimator, is proposed.
  arXiv  Detail & Related papers  (2020-05-07T17:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.