Hyperparameter Selection for Subsampling Bootstraps
- URL: http://arxiv.org/abs/2006.01786v2
- Date: Thu, 13 Jan 2022 07:57:40 GMT
- Title: Hyperparameter Selection for Subsampling Bootstraps
- Authors: Yingying Ma and Hansheng Wang
- Abstract summary: A subsampling method like BLB serves as a powerful tool for assessing the quality of estimators for massive data.
The performance of the subsampling methods are highly influenced by the selection of tuning parameters.
We develop a hyperparameter selection methodology, which can be used to select tuning parameters for subsampling methods.
Both simulation studies and real data analysis demonstrate the superior advantage of our method.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Massive data analysis becomes increasingly prevalent, subsampling methods
like BLB (Bag of Little Bootstraps) serves as powerful tools for assessing the
quality of estimators for massive data. However, the performance of the
subsampling methods are highly influenced by the selection of tuning parameters
( e.g., the subset size, number of resamples per subset ). In this article we
develop a hyperparameter selection methodology, which can be used to select
tuning parameters for subsampling methods. Specifically, by a careful
theoretical analysis, we find an analytically simple and elegant relationship
between the asymptotic efficiency of various subsampling estimators and their
hyperparameters. This leads to an optimal choice of the hyperparameters. More
specifically, for an arbitrarily specified hyperparameter set, we can improve
it to be a new set of hyperparameters with no extra CPU time cost, but the
resulting estimator's statistical efficiency can be much improved. Both
simulation studies and real data analysis demonstrate the superior advantage of
our method.
Related papers
- Scaling Exponents Across Parameterizations and Optimizers [94.54718325264218]
We propose a new perspective on parameterization by investigating a key assumption in prior work.
Our empirical investigation includes tens of thousands of models trained with all combinations of threes.
We find that the best learning rate scaling prescription would often have been excluded by the assumptions in prior work.
arXiv Detail & Related papers (2024-07-08T12:32:51Z) - Targeted Efficient Fine-tuning: Optimizing Parameter Updates with Data-Driven Sample Selection [8.626228174152365]
Iterative Range Decreasing (IRD) algorithm is proposed to optimize the sample- parameter pair selection in FISH Mask.
We demonstrate the effectiveness and rationality of proposed strategy by conducting experiments on GLUE benchmark.
arXiv Detail & Related papers (2024-03-13T12:50:23Z) - Efficient and Robust Bayesian Selection of Hyperparameters in Dimension
Reduction for Visualization [0.0]
We introduce an efficient and robust auto-tuning framework for hyper parameter selection in dimension reduction (DR) algorithms.
Our approach enables efficient hyper parameter selection with multi-objective trade-offs and allows us to perform data-driven analysis.
We evaluate our results on various synthetic and real-world datasets using multiple quality metrics.
arXiv Detail & Related papers (2023-06-01T05:36:22Z) - On the Effectiveness of Parameter-Efficient Fine-Tuning [79.6302606855302]
Currently, many research works propose to only fine-tune a small portion of the parameters while keeping most of the parameters shared across different tasks.
We show that all of the methods are actually sparse fine-tuned models and conduct a novel theoretical analysis of them.
Despite the effectiveness of sparsity grounded by our theory, it still remains an open problem of how to choose the tunable parameters.
arXiv Detail & Related papers (2022-11-28T17:41:48Z) - Optimization of Annealed Importance Sampling Hyperparameters [77.34726150561087]
Annealed Importance Sampling (AIS) is a popular algorithm used to estimates the intractable marginal likelihood of deep generative models.
We present a parameteric AIS process with flexible intermediary distributions and optimize the bridging distributions to use fewer number of steps for sampling.
We assess the performance of our optimized AIS for marginal likelihood estimation of deep generative models and compare it to other estimators.
arXiv Detail & Related papers (2022-09-27T07:58:25Z) - Sparse high-dimensional linear regression with a partitioned empirical
Bayes ECM algorithm [62.997667081978825]
We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression.
Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates.
The proposed approach is implemented in the R package probe.
arXiv Detail & Related papers (2022-09-16T19:15:50Z) - AUTOMATA: Gradient Based Data Subset Selection for Compute-Efficient
Hyper-parameter Tuning [72.54359545547904]
We propose a gradient-based subset selection framework for hyper- parameter tuning.
We show that using gradient-based data subsets for hyper- parameter tuning achieves significantly faster turnaround times and speedups of 3$times$-30$times$.
arXiv Detail & Related papers (2022-03-15T19:25:01Z) - An Asymptotically Optimal Multi-Armed Bandit Algorithm and
Hyperparameter Optimization [48.5614138038673]
We propose an efficient and robust bandit-based algorithm called Sub-Sampling (SS) in the scenario of hyper parameter search evaluation.
We also develop a novel hyper parameter optimization algorithm called BOSS.
Empirical studies validate our theoretical arguments of SS and demonstrate the superior performance of BOSS on a number of applications.
arXiv Detail & Related papers (2020-07-11T03:15:21Z) - Automatic Setting of DNN Hyper-Parameters by Mixing Bayesian
Optimization and Tuning Rules [0.6875312133832078]
We build a new algorithm for evaluating and analyzing the results of the network on the training and validation sets.
We use a set of tuning rules to add new hyper-parameters and/or to reduce the hyper- parameter search space to select a better combination.
arXiv Detail & Related papers (2020-06-03T08:53:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.