A Genetic Algorithm-based Framework for Learning Statistical Power
Manifold
- URL: http://arxiv.org/abs/2209.00215v1
- Date: Thu, 1 Sep 2022 04:15:42 GMT
- Title: A Genetic Algorithm-based Framework for Learning Statistical Power
Manifold
- Authors: Abhishek K. Umrawal, Sean P. Lane, and Erin P. Hennes
- Abstract summary: We propose a novel genetic algorithm-based framework for learning statistical power manifold.
For a multiple linear regression $F$-test, we show that the proposed algorithm learns the statistical power manifold much faster as compared to a brute-force approach.
- Score: 1.7205106391379026
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Statistical power is a measure of the goodness/strength of a hypothesis test.
Formally, it is the probability of detecting an effect, if there is a true
effect present to detect. Hence, optimizing statistical power as a function of
some parameters of a hypothesis test is desirable. However, for most hypothesis
tests, the explicit functional form of statistical power as a function of those
parameters is unknown but calculating statistical power for a given set of
values of those parameters is possible using simulated experiments. These
simulated experiments are usually computationally expensive. Hence, developing
the entire statistical power manifold using simulations can be very
time-consuming. Motivated by this, we propose a novel genetic algorithm-based
framework for learning statistical power manifold. For a multiple linear
regression $F$-test, we show that the proposed algorithm/framework learns the
statistical power manifold much faster as compared to a brute-force approach as
the number of queries to the power oracle is significantly reduced. We also
show that the quality of learning the manifold improves as the number of
iterations increases for the genetic algorithm.
Related papers
- Nonparametric independence tests in high-dimensional settings, with applications to the genetics of complex disease [55.2480439325792]
We show how defining adequate premetric structures on the support spaces of the genetic data allows for novel approaches to such testing.
For each problem, we provide mathematical results, simulations and the application to real data.
arXiv Detail & Related papers (2024-07-29T01:00:53Z) - Precise Error Rates for Computationally Efficient Testing [75.63895690909241]
We revisit the question of simple-versus-simple hypothesis testing with an eye towards computational complexity.
An existing test based on linear spectral statistics achieves the best possible tradeoff curve between type I and type II error rates.
arXiv Detail & Related papers (2023-11-01T04:41:16Z) - Hypothesis Testing and Machine Learning: Interpreting Variable Effects
in Deep Artificial Neural Networks using Cohen's f2 [0.0]
Deep artificial neural networks show high predictive performance in many fields.
But they do not afford statistical inferences and their black-box operations are too complicated for humans to comprehend.
This article extends current XAI methods and develops a model agnostic hypothesis testing framework for machine learning.
arXiv Detail & Related papers (2023-02-02T20:43:37Z) - Sequential Permutation Testing of Random Forest Variable Importance
Measures [68.8204255655161]
It is proposed here to use sequential permutation tests and sequential p-value estimation to reduce the high computational costs associated with conventional permutation tests.
The results of simulation studies confirm that the theoretical properties of the sequential tests apply.
The numerical stability of the methods is investigated in two additional application studies.
arXiv Detail & Related papers (2022-06-02T20:16:50Z) - Nonparametric Conditional Local Independence Testing [69.31200003384122]
Conditional local independence is an independence relation among continuous time processes.
No nonparametric test of conditional local independence has been available.
We propose such a nonparametric test based on double machine learning.
arXiv Detail & Related papers (2022-03-25T10:31:02Z) - Computational-Statistical Gaps in Reinforcement Learning [23.517741855454044]
We show a reduction from Unique-Sat, where we convert a CNF formula into an MDP Hypothesis with deterministic transitions, constant number of actions and low dimensional linear optimal value functions.
This result also exhibits the first computational-statistical gap in reinforcement learning with linear function approximation.
arXiv Detail & Related papers (2022-02-11T04:48:35Z) - PowerGraph: Using neural networks and principal components to
multivariate statistical power trade-offs [0.0]
A priori statistical power estimation for planned studies with multiple model parameters is inherently a multivariate problem.
Explicit solutions in such cases are either impractical or impossible to solve, leaving researchers with the prevailing method of simulating power.
This paper explores the efficient estimation and graphing of statistical power for a study over varying model parameter combinations.
arXiv Detail & Related papers (2021-12-29T19:06:29Z) - Significance tests of feature relevance for a blackbox learner [6.72450543613463]
We derive two consistent tests for the feature relevance of a blackbox learner.
The first evaluates a loss difference with perturbation on an inference sample.
The second splits the inference sample into two but does not require data perturbation.
arXiv Detail & Related papers (2021-03-02T00:59:19Z) - Leveraging Global Parameters for Flow-based Neural Posterior Estimation [90.21090932619695]
Inferring the parameters of a model based on experimental observations is central to the scientific method.
A particularly challenging setting is when the model is strongly indeterminate, i.e., when distinct sets of parameters yield identical observations.
We present a method for cracking such indeterminacy by exploiting additional information conveyed by an auxiliary set of observations sharing global parameters.
arXiv Detail & Related papers (2021-02-12T12:23:13Z) - Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples.
We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
arXiv Detail & Related papers (2020-05-22T22:30:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.